diff --git a/docs/imagecustomizer/api/cli.md b/docs/imagecustomizer/api/cli.md index 6ab3811d51..af126cd999 100644 --- a/docs/imagecustomizer/api/cli.md +++ b/docs/imagecustomizer/api/cli.md @@ -60,6 +60,16 @@ The file path to write the final customized image to. Added in v0.3. +If the `--output-format` is set to `pxe`, this can be set to either: +- a directory where the customized PXE artifacts will be placed. +- a `.tar.gz` file name where the PXE artifacts will be compressed and stored. + +## --output-path=FILE-PATH + +An alias to `--output-image-file` + +Added in v0.15. + ## --output-image-format=FORMAT Required, unless @@ -70,7 +80,7 @@ value is used. The image format of the final customized image. -Options: vhd, vhd-fixed, vhdx, qcow2, raw, iso, and [cosi](./cosi.md). +Options: vhd, vhd-fixed, vhdx, qcow2, raw, iso, pxe, and [cosi](./cosi.md). The vhd-fixed option outputs a fixed size VHD image. This is the required format for VMs in Azure. @@ -145,15 +155,6 @@ installation. Added in v0.3. -## --output-pxe-artifacts-dir - -Create a folder containing the artifacts to be used for PXE booting. - -For an overview of Image Customizer support for PXE, see the -[PXE support page](../concepts/pxe.md). - -Added in v0.8. - ## --log-level=LEVEL Default: `info` diff --git a/docs/imagecustomizer/api/cli/inject-files.md b/docs/imagecustomizer/api/cli/inject-files.md index c06b107232..cf3db4a084 100644 --- a/docs/imagecustomizer/api/cli/inject-files.md +++ b/docs/imagecustomizer/api/cli/inject-files.md @@ -60,6 +60,12 @@ is modified at `--image-file`. Added in v0.14. +## --output-path=FILE-PATH + +An alias to `--output-image-file` + +Added in v0.15. + ## --output-image-format=FORMAT Optional. diff --git a/docs/imagecustomizer/api/configuration/iso.md b/docs/imagecustomizer/api/configuration/iso.md index 60359cca7a..0f02a3e121 100644 --- a/docs/imagecustomizer/api/configuration/iso.md +++ b/docs/imagecustomizer/api/configuration/iso.md @@ -4,7 +4,8 @@ parent: Configuration # iso type -Specifies the configuration for the generated ISO media. +Specifies the configuration for the generated ISO image when the `--output-format` +is set to `iso`. Example: @@ -17,9 +18,11 @@ iso: kernelCommandLine: extraCommandLine: - rd.info + + initramfsType: bootstrap ``` -See also: [ISO Support](../../concepts/iso.md) +See also the [ISO Support](../../concepts/iso.md) page. ## kernelCommandLine [[kernelCommandLine](./kernelcommandline.md)] @@ -32,3 +35,28 @@ Added in v0.3. Adds files to the ISO. Added in v0.7. + +## initramfsType [string] + +Specifies the initramfs type to generate and include in the ISO image. + +Supported options: + +- `bootstrap`: Creates a minimal Dracut-based initramfs image that later + transitions to the full OS. The full OS is packaged in a separate image + and is included on the media. This option allows the generated ISO to boot + on hardware that has memory restrictions on the initramfs image size. +- `full-os`: Creates a full OS initramfs image. + +The default is `initramfsType` value is `bootstrap`. + +Note that SELinux cannot be enabled if `initramfsType` is set to `full-os`. + +Example: + +```yaml +iso: + initramfsType: full-os +``` + +Added in v0.15. \ No newline at end of file diff --git a/docs/imagecustomizer/api/configuration/pxe.md b/docs/imagecustomizer/api/configuration/pxe.md index baf6c96928..900e8f3651 100644 --- a/docs/imagecustomizer/api/configuration/pxe.md +++ b/docs/imagecustomizer/api/configuration/pxe.md @@ -4,76 +4,116 @@ parent: Configuration # pxe type -Specifies the PXE-specific configuration for the generated OS artifacts. +Specifies the configuration for the generated PXE artifacts when the `--output-format` +is set to `pxe`. -Added in v0.8. +Note that when the `--output-format` is set to `pxe`, `--output-image-file` can +be set to either: +- a directory where the customized PXE artifacts will be placed. +- a `.tar.gz` file name where the PXE artifacts will be compressed and stored. -## isoImageBaseUrl [string] +Example: -Specifies the base URL for the ISO image to download at boot time. The Azure -Linux Image Customizer will append the output image name to the specified base -URL to form the full URL for downloading the image. The output image name is -specified on the command-line using the `--output-image file` argument (see the - [command-line interface](../cli.md) document for more details). +```yaml +pxe: + additionalFiles: + - source: files/a.txt + destination: /a.txt -This can be useful if the ISO image name changes with each build and the -script deploying the artifacts to the PXE server does not update grub.cfg with -the ISO image name. + kernelCommandLine: + extraCommandLine: + - rd.info -For example, + initramfsType: bootstrap + + bootstrapBaseUrl: http://my-pxe-server +``` + +See also the [PXE Support](../../concepts/pxe.md) page. + +## kernelCommandLine [[kernelCommandLine](./kernelcommandline.md)] + +Specifies extra kernel command line options. + +Added in v0.15. + +## additionalFiles [[additionalFile](./additionalfile.md)[]] + +Adds files to the PXE artifacts. + +Added in v0.15. -- If the user has the following content in the configuration file: +## initramfsType [string] - ```yaml - pxe: - isoImageBaseUrl: http://hostname-or-ip/iso-publish-path - ``` +Specifies the initramfs type to generate and include in the PXE artifacts. -- and specifies the following on the command line: +Supported options: - ```bash - sudo imagecustomizer \ - --image-file "./input/azure-linux.vhdx" \ - --config-file "./input/customization-config.yaml" \ - --rpm-source "./input/rpms" \ - --build-dir "./build" \ - --output-image-format "iso" \ - --output-image-file "./build/output/output.iso" \ - --output-pxe-artifacts-dir "./build/output/pxe-artifacts" - ``` +- `bootstrap`: Creates a minimal Dracut-based initramfs image that later + transitions to the full OS. The full OS is packaged in a separate image + and is included on in the PXE artifacts folder for the initramfs to find it. + This option allows the generated PXE artifacts to boot on hardware that has + memory restrictions on the initramfs image size. +- `full-os`: Creates a full OS initramfs image. -- then, during PXE booting, the ISO image will be downloaded from: +The default is `initramfsType` value is `full-os`. - ```bash - http://hostname-or-ip/iso-publish-path/output.iso - ``` +Note that SELinux cannot be enabled if `initramfsType` is set to `full-os`. -This field is mutually exclusive with `isoImageFileUrl`. +Example: + +```yaml +pxe: + initramfsType: bootstrap +``` + +Added in v0.15. + +## bootstrapBaseUrl [string] + +Specifies the base URL for the bootstrap ISO image to download at boot time. +The bootstrap ISO image is generated by the Image Customizer along with the +rest of the PXE artifacts. The Azure Linux Image Customizer will append the +default image name (`image.iso`) to the specified base URL to form the full URL +for downloading the image. + +For example, if the user has the following content in the configuration file: +```yaml +pxe: + bootstrapBaseUrl: http://hostname-or-ip/iso-publish-path +``` + +Then the download URL becomes `http://hostname-or-ip/iso-publish-path/image.iso`. + +This field is mutually exclusive with `bootstrapFileUrl`. For an overview of Image Customizer support for PXE, see the -[PXE support page](../../concepts/pxe.md). +[PXE support](../../concepts/pxe.md) page. -Added in v0.8. +Added in v0.15. -## isoImageFileUrl [string] +## bootstrapFileUrl [string] -Specifies the URL of the ISO image to download at boot time. -The ISO image must be a LiveOS ISO image generated by the Azure Linux Image -Customizer. The booting process will pivot to the root file system embedded -in the ISO image after downloading it. +Specifies the URL of the bootstrap ISO image to download at boot time. +The bootstrap ISO image is generated by the Image Customizer along with the rest +of the PXE artifacts. The generated ISO will always be placed at the root of the +PXE artifacts folder (archive) and will be have the same name as the file name +specified by `bootstrapFileUrl`. -PXE Configuration Example: +For example, +```yaml +pxe: + bootstrapFileUrl: http://hostname-or-ip/iso-publish-path/my-bootstrap.iso +``` -- ```yaml - pxe: - isoImageFileUrl: http://hostname-or-ip/iso-publish-path/my-liveos.iso - ``` +A file named `my-bootstrap.iso` will be placed at the root of the PXE artifacts +folder. The supported download protocols are: nfs, http, https, ftp, torent, tftp. -This field is mutually exclusive with `isoImageBaseUrl`. +This field is mutually exclusive with `bootstrapBaseUrl`. For an overview of Image Customizer support for PXE, see the -[PXE support page](../../concepts/pxe.md). +[PXE support](../../concepts/pxe.md) page. -Added in v0.8. +Added in v0.15. diff --git a/docs/imagecustomizer/concepts/README.md b/docs/imagecustomizer/concepts/README.md index dea82142d8..919cbefb24 100644 --- a/docs/imagecustomizer/concepts/README.md +++ b/docs/imagecustomizer/concepts/README.md @@ -12,6 +12,7 @@ provides basic functionality, more comprehensive documentation is provided for t following scenarios: * Image History: Record the customization history of images. See [Image History](./imagehistory.md) for more info +* Live OS: What is a Live OS? More details in [Live OS](./liveos.md) * ISO: Live-ISO creation. More details in [ISO Support](./iso.md) * PXE: Creating a PXE bootable image. More details in [PXE Support](./pxe.md) * Verity protected images: [Guidelines for creating a verity-protected root filesystem](./verity.md) diff --git a/docs/imagecustomizer/concepts/iso.md b/docs/imagecustomizer/concepts/iso.md index e0a86254e8..fee499ace5 100644 --- a/docs/imagecustomizer/concepts/iso.md +++ b/docs/imagecustomizer/concepts/iso.md @@ -1,99 +1,29 @@ --- parent: Concepts title: ISO Support -nav_order: 2 +nav_order: 3 --- -# Image Customizer LiveOS ISO Support +# Image Customizer Live OS ISO Support ## Overview The Image Customizer can customize an input image and package the output as a -LiveOS iso image. The LiveOS iso image is a bootable image that boots into a -rootfs image included on the iso media without the need to have anything -pre-installed on the target machine. The rootfs is kept read-only and an overlay -in memory is created so that processes can successfully write state. +[Live OS](./liveos.md) iso image. A Live OS iso image is a bootable image that +boots into a root file system included on the iso media without the need to +have anything pre-installed on the target machine. -## Creating a LiveOS ISO +## Creating a Live OS ISO -The input image can be a full disk image (vhd/vhdx/qcow2/raw) or another LiveOS iso image. +The input image can be a full disk image (vhd/vhdx/qcow2/raw) or previously +generated Live OS iso image. -To generate a LiveOS ISO, set the `--output-image-format` parameter to `iso`. +To generate a Live OS iso, set the `--output-image-format` parameter to `iso`. More info can be found at [Creating a LiveOS ISO how-to guide](../how-to/live-iso.md) -## Input Image Layout Assumptions - -### Full Disk Image - -The input full disk image must satisfy the following requirements in order for -the Azure Linux Image Cuztomizer to be able to generate an iso image out of it: - -- File layout (after all partitions have been mounted): - - `/boot/grub2/grub.cfg` must exist and is the 'main' grub configuration (not - a redirection grub configuration file for example). - - The bootloader and the shim must exist under the `/boot` folder or any of - its subdirectories. - - For x64, `bootx64.efi` and `grubx64.efi` (or `grubx64-noprefix.efi`). - - For ARM64, `bootaa64.efi` and `grubaa64.efi` (or `grubaa64-noprefix.efi`). - - All grub configurations and related files must be stored under the `/boot` - folder. For example, grub.cfg cannot reference files outside that folder. - If it does, those referenced files will not be copied to the iso and may - cause grub to fail booting the desired operating system. - -### LiveOS ISO Image - -The input LiveOS iso image must be previously generated by the Azure Linux Image -Customizer. - -## How LiveOS ISO Customization Works - -Customizations are made to the input rootfs as usual using the Azure Linux Image -Customizer, and will naturally carry over into the LiveOS iso image. This -includes customizations for kernel modules, dracut, and other early boot-time -actions. - -While converting the input full disk image into a LiveOS iso involves copying -almost all the artifacts unchanged - some artifacts are changed as follows: - -- `grub.cfg` is modified by updating the kernel command-line arguments as - follows: - - the root is updated to the LiveOS root file system image. - - the LiveOS dracut parameters are appended. - - the user-specified new parameters are appended. - -- `/etc/fstab` is dropped from the rootfs as it typically conflicts with the - overlay setup required by the LiveOS. - -- `initrd.img` is regenerated to serve the LiveOS boot flow. This should have - no impact as long as the included rootfs is where the original `initrd.img` - was generated using `dracut`. The use of such rootfs guarantees that the same - dracut configuration that got used before will be re-used again when we are - re-generating the `initrd.img` and hence the original behavior is retained. - -## Limitations - -The current implementation for the LiveOS iso does not support the following: - -- Persistent storage: The machine loses all its state on reboot or shutdown. - -- dm-verity: The iso image cannot run dm-verity for the LiveOS partitions. - -- Disk layout: There is always one disk generated when an `iso` output format is - specified. - -- Non-Azure Linux official images: Image generated by tools other than the Azure Linux - toolkit may fail to be converted to a LiveOS iso. This is due to certain assumptions - on the input image layout that may not be satisfied. See the section titled - [Input Image Layout Assumptions](#input-image-layout-assumptions) for more details. - -## ISO Specific Customizations - -- The user can specify one or more files to be copied to the iso media. -- The user can add kernel parameters. - -For a full list of capabilities, see Image Customizer's iso -configuration section: [Config.ISO](../api/configuration/iso.md). +For a full list of capabilities, see [ISO configuration](../api/configuration/iso.md) +page ## cloud-init Support @@ -110,7 +40,7 @@ cloud-init data files). ### Examples -#### Example 1 - Placing cloud-init data directly within the ISO file system +#### Example 1 - Placing cloud-init data directly within the iso file system ```yaml scripts: diff --git a/docs/imagecustomizer/concepts/liveos.md b/docs/imagecustomizer/concepts/liveos.md new file mode 100644 index 0000000000..80d9fb00e8 --- /dev/null +++ b/docs/imagecustomizer/concepts/liveos.md @@ -0,0 +1,163 @@ +--- +parent: Concepts +title: Live OS +nav_order: 2 +--- + +# Image Customizer Live OS Support + +## What is a Live OS? + +A Live OS is when a system is booted from removable media (for example, an ISO +or a USB) or from the network (for example, using PXE). In such flows, the OS +file system is not installed to persistent storage on the host prior to fully +booting the system. + +While the Live OS itself is not a concrete image format, it is a composition +and a flow that is typically used when producing an ISO image or PXE artifacts +(folder or tar.gz). + +The Live OS is typically composed of the following core artifacts: +- the boot loader (the shim and something like grub) +- the boot loader configuration +- the kernel image +- the initrd image +- additional files (optional) + +The flow is as follows: +- The firmware starts the bootloader. +- The bootloader reads its configuration and determines where the kernel/initrd + are. +- The bootloader loads the kernel and initrd in memory. +- The bootloader transfers control to the kernel and the initrd is attached. +- The kernel initializes itself and then runs the initrd initialization entry + point. +- The initrd initialization entry point starts systemd (or other processes + depending on the configuration) +- One of the initrd processes can optionally locate another root file system and + pivots to it if desired. + +## Support in Image Customizer + +The Image Customizer can take an input image (qcow, vhd(x), prerviously generated +Live OS ISO) produces new ISO images and PXE artifacts. For details, see: +- [ISO](./iso.md) +- [PXE](./pxe.md) + +### Limitations + +The current implementation for the Live OS does not support the following: + +- Persistent storage: The machine loses all its state on reboot or shutdown. + +- dm-verity: The image cannot run dm-verity for the Live OS partitions. + +- Disk layout: There is always one disk generated when an Live OS output format + is specified. + +- Non-Azure Linux official images: Image generated by tools other than the Azure Linux + toolkit may fail to be converted to a LiveOS iso. This is due to certain assumptions + on the input image layout that may not be satisfied. See the section titled + [Input Image Layout Assumptions](#input-image-layout-assumptions) for more details. + +### Input Image Requirements + +The input image can come in many formats: +- an image with multiple partitions. +- an ISO image. + +The Image Customizer takes all supported input formats and normalize them +into a full OS layout (mount partitions, extract squashfs contents, etc.). +Then, it will proceed to customize it and then look for following: + +- `/boot/grub2/grub.cfg` must exist and is the 'main' grub configuration (not + a redirection grub configuration file for example). +- The bootloader and the shim must exist under the `/boot` folder or one of + its subdirectories. + - For x64, `bootx64.efi` and `grubx64.efi` (or `grubx64-noprefix.efi`). + - For ARM64, `bootaa64.efi` and `grubaa64.efi` (or `grubaa64-noprefix.efi`). +- All grub configurations and related files must be stored under the `/boot` + folder. For example, grub.cfg cannot reference files outside that folder. + If it does, those referenced files will not be copied to the iso and may + cause grub to fail booting the desired operating system. + +### OS Modifications + +The user can customize the full OS using the Image Customizer as usual (using +the `os` section in the configuration). All those customizations will carry over +into the Live OS naturally. This includes customizations for kernel modules, +dracut, and other early boot-time actions. + +While converting the input full disk image into a Live OS involves copying +almost all the artifacts unchanged - some artifacts are changed as follows: + +- `grub.cfg` is modified by updating the kernel command-line arguments: + - If the root file system is bootstrapped (`initramfsType=boostrap`), the + `root` value is updated to the location of the bootstrap root file system + image (for example: `root=live:/liveos/image.img`). + - If the root file system is the same as the initramfs (`initramfsType=full-os`), + the `root` parameter is removed from the kernel parameters. + - The user-specified iso kernel parameters are appended. + +- `/etc/fstab` is removed from the root file system as it typically conflicts + with the overlay setup required by the LiveOS. + +- `initrd.img` is regenerated to serve the Live OS boot flow. + - If the initramfs type is set to `bootstrap`, Dracut will regenerate the + `initrd.img` using the input root file system and the Dracut configurations + already there. This allows the user to customize the initramfs by dropping + Dracut configurations on the root file system during the OS customization + phase, and then when the Image Customizer runs Dracut to regenerate the + initrd image, the newly placed/updated configuration will take effect (this + is the typical way of customizing the initramfs when using Dracut to generate + it). + - If the initramfs type is `full-os`, the Image Customizer copies all the mounted + partitions (minus the `/boot` folder) to the initramfs image. This allows the + user to customize the initramfs by using the `os` sections to install/uninstall + packages, copy files, etc as usual. + +- SELinux must be disabled if the full OS is initrd. This is because the kernel + expects the initrd image to be a CPIO archive, and that format does not + support extended attributes - which are necessary to hold the SELinux labels. + +### Initramfs Contents + +The user can decide whether the full OS file system will be embedded in the +initrd image itself or stored in a separate image (in which case, a process +on the initrd image will need to find it, and pivot to it). + +The main difference between the two configurations is: +- When the initrd image is the full OS: + - the initrd image size is larger and may not meet the memory restriction on + certain hardware skus (leading to an "out of memory" grub error). However, + this is much simpler to deploy to a PXE server since the full OS content + will be served using the PXE protocol without any extra setup. + - initrd is customized using the Image Customizer configuration constructs + directly (install packages, copy files, etc). + - selinux is not supported (mainly because CPIO does not support extended file + system attributes). +- When the initrd image is a bootstrap image: + - the initrd image size will be much smaller (~30MB) - which solves the memory + restriction issue on affected hardware models. + - The downside is that extra PXE environment setup is necessary (like setting up + a server to host the bootstrapped extra file and server it). + - initrd is customized using Dracut configuration files. The user will need to + craft those files, use Image Customizers to place them in the right locations + on the root file system so that when Dracut is run to generate a new initrd, + it will take them into consideration. + - selinux can be enabled. + +The user can specify either configuration using the `initramfsType` property. + +### Bootstrap Implementation + +- ISO support: + - The bootstrap support is implemented using Dracut's `dmsquash` module. The + module expects the full OS image to be a squashfs image placed under + `/liveos/rootfs.img`. Once it is found, it will be mounted, an overlay will + be created, and then the kernel will pivot to it. +- PXE Support: + - To support the Live OS bootstrap PXE scenario, Dracut's `livenet` module + downloads the bootstrapped image (expects an .ISO), and then passes it to the + `dmsquash` module. To enable this Dracut flow, the kernel command-line must + contain a `root` parameter on the form `root=live:liveos-iso-url`. diff --git a/docs/imagecustomizer/concepts/pxe.md b/docs/imagecustomizer/concepts/pxe.md index c54eb753fe..5f570df584 100644 --- a/docs/imagecustomizer/concepts/pxe.md +++ b/docs/imagecustomizer/concepts/pxe.md @@ -1,7 +1,7 @@ --- parent: Concepts title: PXE Support -nav_order: 3 +nav_order: 4 --- # Image Customizer PXE Support @@ -15,12 +15,12 @@ hosts and also centralizes the deployment configuration to a single server. One way of enabling such setup is using the PXE (Preboot eXecution Environment) Boot protocol. The user can setup a server with all the OS artifacts, a DHCP endpoint, and a tftp connection endpoint. When a client machine is powered on, -and its firmware will look for a DHCP server on the same network and find the +its firmware will look for a DHCP server on the same network and will find the one configured by the user. The DHCP server will serve information about the tftp endpoint to the client, and the client firmware can then proceed with retrieving the OS artifacts over -tftp, then loading them into memory, and finally handing control over to the +tftp, loading them into memory, and finally handing control over to the loaded OS. The tftp protocol expects certain artifacts to be present on the server: @@ -30,108 +30,73 @@ The tftp protocol expects certain artifacts to be present on the server: - the kernel image. - the initrd image. -Once retrieved, the boot loader is run. Then the boot loader reads the -boot loader configuration and then transfers control over to the kernel image -with the retrieved initrd image as its file system. +Once the PXE client retrieves those artifacts, the boot loader is run and it +reads the boot loader configuration. It then transfers control over to the +kernel image with the retrieved initrd image as its file system. The initrd image is customized to perform the next set of tasks now that an OS is running. The tasks can range from just running some local scripts all the way to installing another OS. -## LiveOS ISOs and PXE Support +## PXE Support in The Image Customizer -A [LiveOS ISO](./iso.md) image is a bootable ISO image that runs all the necessary -components from memory (i.e. does not need to install anything to the host -persistent storage). +The Image Customizer can take an input image, customize the OS, and produce +the artifacts necessary to power the PXE flow described above. -The necessary components can be either embedded into the initrd image itself -or embedded into a separate 'rootfs' image (to allow much smaller -initrd images). If separate, then, the initrd image must be configured with an -agent that will look for the rootfs image, and transition control over to the -rootfs at boot time. Dracut provides the `dmsquash-live` module which managed this transition from -the initrd image over to the rootfs image. +In addition to customizing the OS contents, the user can also decide how it +will be packaged. There are two supported options: +- The initrd image is the full OS. +- The initrd image is a bootstrap image with minimal contents, and the full + OS is stored in a separate image. -The **Image Customizer** produces such [LiveOS ISO](./iso.md) images. A typical -image holds the following artifacts: +There are pros and cons for each configuration - see the [Live OS](./liveos.md) +page for details. -- the boot loader (the shim and something like grub) -- the boot loader configuration -- the kernel image -- the initrd image -- the rootfs image -- other user defined artifacts (optional) +When the bootstrap configuration is selected, The bootstrapped image is a +bootable ISO (`image.iso`) containing the same PXE configuration (same bootloader +configuration, same kernel, same full OS file system, etc.). This can be very +handy when testing the PXE configuration without having to setup a PXE +environment. -Note that the first 4 artifacts are what is necessary to get an OS kernel up -and running in a network boot scenario. What remains for a successful booting -of a LiveOS over the network is to make the rootfs image available for the final -transition (during the initrd phase). - -Dracut enables that entire flow through the use of the `livenet` module - where -it inspects the `root=live:liveos-iso-url` kernel parameter from the boot loader -config file, and if it recognizes the `liveos-iso-url` protocol, it downloads -the ISO, and then proceeds to pivot to the embedded rootfs image. - -The user can customize the rootfs using the Image Customizer as -usual. In case of additional artifacts that need downloading, the user can -install a daemon on the rootfs which will run when control is transferred to -the rootfs image and download any additional items. +If the user needs to download additional artifacts after booting, the user can +implement a user-space solution that will perform the download tasks. ## Creating and Deploying PXE Boot Artifacts -The Image Customizer produces [LiveOS ISO](./iso.md) images that are also PXE -bootable. So, the user can simply create an ISO image as usual, and the output -can be taken and deployed to a PXE server. - -To make the deployment of the generated artifacts easier for the user, the -Image Customizer offers the following configurations: +The Image Customizer can create the PXE artifacts by simply setting the `--output-image-format` +parameter to `pxe` on the command-line. The output can be either a folder containing +the artifacts, or a tar.gz with the same content. -- In the input configuration, there is a `pxe` node under which the user can - configure PXE related properties - like the URL of the LiveOS ISO image to - download (note that this image is the same image being built). - See the [Image Customizer configuration](../api/configuration/pxe.md) - page for more information. -- When invoking the Image Customizer, the user can also elect to - export the artifacts to a local folder. - See the [Image Customizer command line](../api/cli.md#--output-pxe-artifacts-dir) - page for more information. +For additional details, see the [PXE Configuration](../api/configuration/pxe.md) +page. -Below is a list of required artifacts and where on the PXE server they should +Below is a list of the core artifacts and where on the PXE server they should be deployed: ``` -ISO media layout artifacts local folder target on PXE server ------------------------ ------------------------ ------------------------------ -|- efi | - |- boot | | - |- bootx64.efi |- bootx64.efi |- bootx64.efi - |- grubx64.efi |- grubx64.efi |- grubx64.efi -|- boot |- boot |- boot - |- grub2 |- grub2 |- grub2 - |- grub-pxe.cfg |- grub.cfg |- grub.cfg - |- grubenv |- grubenv |- grubenv - |- grub.cfg - |- vmlinuz |- vmlinuz |- vmlinuz - |- initrd.img |- initrd.img |- initrd.img - - -|- other-user-artifacts |- other-user-artifacts |- other-user-artifacts - |- .iso |- .iso +artifacts output folder target on PXE server +------------------------ ------------------------------ +| +|- bootx64.efi |- bootx64.efi +|- grubx64.efi |- grubx64.efi +|- boot |- boot + |- grub2 |- grub2 + |- grub.cfg |- grub.cfg + |- grubenv |- grubenv +|- vmlinuz |- vmlinuz +|- initrd.img |- initrd.img +| +| +|- other-user-artifacts |- other-user-artifacts +|- image.iso |- image.iso ``` Notes: -- Note that the `/boot/grub2/grub.cfg` file in the ISO media is not used for - PXE booting. Instead, the `/boot/grub2/grub-pxe.cfg` gets renamed to `grub.cfg` - and is used instead. - `yyyy` can be any protocol supported by Dracut's `livenet` module (i.e tftp, http, etc). -- The ISO image file location under the server root is customizable - - but it must be such that its URL matches what is specified in the grub.cfg - `root=live:`. -- While the core OS artifacts (the bootloader, its configuration, the kernel, - initrd image, and rootfs image) will be downloaded and used automatically, - the user will need to independently implement a way to download any - additional artifacts. For example, the user can implement a daemon (and place - it on the root file system) that will reach out and download the additional - artifacts when it is up and running. The daemon can be configured with where - to download the artifacts from, and what to do with them. +- `image.iso` is the bootstrapped image containing the full OS file system. It + is generated by the Image Customizer when the output format is set to `pxe`. +- The bootstrapped ISO image file location under the server root is customizable - + but it must match what is specified in grub.cfg's `root=live:` (configured + through `bootstrapBaseUrl` or `bootstrapFileUrl`). diff --git a/docs/imagecustomizer/concepts/sysext.md b/docs/imagecustomizer/concepts/sysext.md index eb2536642f..024ea3b9e1 100644 --- a/docs/imagecustomizer/concepts/sysext.md +++ b/docs/imagecustomizer/concepts/sysext.md @@ -1,6 +1,6 @@ --- parent: Concepts -nav_order: 6 +nav_order: 7 --- # System extension diff --git a/docs/imagecustomizer/concepts/things-to-avoid.md b/docs/imagecustomizer/concepts/things-to-avoid.md index 45f0061c8f..1ef67b00cf 100644 --- a/docs/imagecustomizer/concepts/things-to-avoid.md +++ b/docs/imagecustomizer/concepts/things-to-avoid.md @@ -1,6 +1,6 @@ --- parent: Concepts -nav_order: 4 +nav_order: 5 --- # Things to avoid diff --git a/docs/imagecustomizer/concepts/verity.md b/docs/imagecustomizer/concepts/verity.md index e0e4f3d8c6..c80d8f4504 100644 --- a/docs/imagecustomizer/concepts/verity.md +++ b/docs/imagecustomizer/concepts/verity.md @@ -1,6 +1,6 @@ --- parent: Concepts -nav_order: 5 +nav_order: 6 --- # Verity Image Recommendations diff --git a/test/vmtests/vmtests/test_min_change.py b/test/vmtests/vmtests/test_min_change.py index ffa7cd6bab..74d68cbd37 100644 --- a/test/vmtests/vmtests/test_min_change.py +++ b/test/vmtests/vmtests/test_min_change.py @@ -283,7 +283,7 @@ def test_min_change_efi_azl2_iso_output( close_list: List[Closeable], ) -> None: azl_release = 2 - config_path = TEST_CONFIGS_DIR.joinpath("iso-os-vm-config.yaml") + config_path = TEST_CONFIGS_DIR.joinpath("iso-bootstrap-vm.yaml") output_format = "iso" run_min_change_test( @@ -302,7 +302,7 @@ def test_min_change_efi_azl2_iso_output( ) -def test_min_change_efi_azl3_iso_output( +def test_min_change_efi_azl3_iso_bootstrap_output( docker_client: DockerClient, image_customizer_container_url: str, core_efi_azl3: Path, @@ -314,7 +314,38 @@ def test_min_change_efi_azl3_iso_output( close_list: List[Closeable], ) -> None: azl_release = 3 - config_path = TEST_CONFIGS_DIR.joinpath("iso-os-vm-config.yaml") + config_path = TEST_CONFIGS_DIR.joinpath("iso-bootstrap-vm.yaml") + output_format = "iso" + + run_min_change_test( + docker_client, + image_customizer_container_url, + core_efi_azl3, + azl_release, + config_path, + output_format, + ssh_key, + test_temp_dir, + test_instance_name, + logs_dir, + libvirt_conn, + close_list, + ) + + +def test_min_change_efi_azl3_iso_full_os_output( + docker_client: DockerClient, + image_customizer_container_url: str, + core_efi_azl3: Path, + ssh_key: Tuple[str, Path], + test_temp_dir: Path, + test_instance_name: str, + logs_dir: Path, + libvirt_conn: libvirt.virConnect, + close_list: List[Closeable], +) -> None: + azl_release = 3 + config_path = TEST_CONFIGS_DIR.joinpath("iso-full-os-vm.yaml") output_format = "iso" run_min_change_test( @@ -346,7 +377,7 @@ def test_min_change_legacy_azl2_iso_output( close_list: List[Closeable], ) -> None: azl_release = 2 - config_path = TEST_CONFIGS_DIR.joinpath("iso-os-vm-config.yaml") + config_path = TEST_CONFIGS_DIR.joinpath("iso-bootstrap-vm.yaml") output_format = "iso" run_min_change_test( @@ -378,7 +409,7 @@ def test_min_change_legacy_azl3_iso_output( close_list: List[Closeable], ) -> None: azl_release = 3 - config_path = TEST_CONFIGS_DIR.joinpath("iso-os-vm-config.yaml") + config_path = TEST_CONFIGS_DIR.joinpath("iso-bootstrap-vm.yaml") output_format = "iso" run_min_change_test( diff --git a/toolkit/tools/imagecustomizer/main.go b/toolkit/tools/imagecustomizer/main.go index 6ecaaf43c6..b230f770ab 100644 --- a/toolkit/tools/imagecustomizer/main.go +++ b/toolkit/tools/imagecustomizer/main.go @@ -22,12 +22,12 @@ import ( type CustomizeCmd struct { BuildDir string `name:"build-dir" help:"Directory to run build out of." required:""` InputImageFile string `name:"image-file" help:"Path of the base Azure Linux image which the customization will be applied to."` - OutputImageFile string `name:"output-image-file" help:"Path to write the customized image to."` - OutputImageFormat string `name:"output-image-format" placeholder:"(vhd|vhd-fixed|vhdx|qcow2|raw|iso|cosi)" help:"Format of output image." enum:"${imageformat}" default:""` + OutputImageFile string `name:"output-image-file" help:"Path to write the customized image artifacts to."` + OutputPath string `name:"output-path" help: "Path to write the customized image artifacts to."` + OutputImageFormat string `name:"output-image-format" placeholder:"(vhd|vhd-fixed|vhdx|qcow2|raw|iso|pxe|cosi)" help:"Format of output image." enum:"${imageformat}" default:""` ConfigFile string `name:"config-file" help:"Path of the image customization config file." required:""` RpmSources []string `name:"rpm-source" help:"Path to a RPM repo config file or a directory containing RPMs."` DisableBaseImageRpmRepos bool `name:"disable-base-image-rpm-repos" help:"Disable the base image's RPM repos as an RPM source."` - OutputPXEArtifactsDir string `name:"output-pxe-artifacts-dir" help:"Create a directory with customized image PXE booting artifacts. '--output-image-format' must be set to 'iso'."` PackageSnapshotTime string `name:"package-snapshot-time" help:"Only packages published before this snapshot time will be available during customization. Supports 'YYYY-MM-DD' or full RFC3339 timestamp (e.g., 2024-05-20T23:59:59Z)."` } @@ -36,6 +36,7 @@ type InjectFilesCmd struct { ConfigFile string `name:"config-file" help:"Path to the inject-files.yaml config file." required:""` InputImageFile string `name:"image-file" help:"Path of the base image to inject files into." required:""` OutputImageFile string `name:"output-image-file" help:"Path to write the injected image to."` + OutputPath string `name:"output-path" help: "Path to write the customized image artifacts to."` OutputImageFormat string `name:"output-image-format" placeholder:"(vhd|vhd-fixed|vhdx|qcow2|raw|iso|cosi)" help:"Format of output image." enum:"${imageformat}" default:""` } @@ -103,9 +104,12 @@ func main() { } func customizeImage(cmd CustomizeCmd) error { + // Todo: either outputimagefile or outpath + if cmd.OutputImageFile == "" { + cmd.OutputImageFile = cmd.OutputPath + } err := imagecustomizerlib.CustomizeImageWithConfigFile(cmd.BuildDir, cmd.ConfigFile, cmd.InputImageFile, - cmd.RpmSources, cmd.OutputImageFile, cmd.OutputImageFormat, cmd.OutputPXEArtifactsDir, - !cmd.DisableBaseImageRpmRepos, cmd.PackageSnapshotTime) + cmd.RpmSources, cmd.OutputImageFile, cmd.OutputImageFormat, !cmd.DisableBaseImageRpmRepos, cmd.PackageSnapshotTime) if err != nil { return err } @@ -114,6 +118,10 @@ func customizeImage(cmd CustomizeCmd) error { } func injectFiles(cmd InjectFilesCmd) error { + // Todo: either outputimagefile or outpath + if cmd.OutputImageFile == "" { + cmd.OutputImageFile = cmd.OutputPath + } err := imagecustomizerlib.InjectFilesWithConfigFile(cmd.BuildDir, cmd.ConfigFile, cmd.InputImageFile, cmd.OutputImageFile, cmd.OutputImageFormat) if err != nil { diff --git a/toolkit/tools/imagecustomizerapi/imageFormatType.go b/toolkit/tools/imagecustomizerapi/imageFormatType.go index 193656c88a..a3b2416e3c 100644 --- a/toolkit/tools/imagecustomizerapi/imageFormatType.go +++ b/toolkit/tools/imagecustomizerapi/imageFormatType.go @@ -18,6 +18,7 @@ const ( ImageFormatTypeQcow2 ImageFormatType = "qcow2" ImageFormatTypeRaw ImageFormatType = "raw" ImageFormatTypeIso ImageFormatType = "iso" + ImageFormatTypePxe ImageFormatType = "pxe" ImageFormatTypeCosi ImageFormatType = "cosi" ) @@ -30,6 +31,7 @@ var supportedImageFormatTypes = []string{ string(ImageFormatTypeQcow2), string(ImageFormatTypeRaw), string(ImageFormatTypeIso), + string(ImageFormatTypePxe), string(ImageFormatTypeCosi), } diff --git a/toolkit/tools/imagecustomizerapi/initramfsImageType.go b/toolkit/tools/imagecustomizerapi/initramfsImageType.go new file mode 100644 index 0000000000..201d5d1b08 --- /dev/null +++ b/toolkit/tools/imagecustomizerapi/initramfsImageType.go @@ -0,0 +1,37 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +package imagecustomizerapi + +import ( + "fmt" + "slices" +) + +type InitramfsImageType string + +const ( + InitramfsImageTypeUnspecified InitramfsImageType = "" + InitramfsImageTypeBootstrap InitramfsImageType = "bootstrap" + InitramfsImageTypeFullOS InitramfsImageType = "full-os" +) + +// supportedInitramfsImageTypes is a list of all non-empty image format types +// defined above. +var supportedInitramfsImageTypes = []string{ + string(InitramfsImageTypeBootstrap), + string(InitramfsImageTypeFullOS), +} + +func (ft InitramfsImageType) IsValid() error { + if ft != InitramfsImageTypeUnspecified && !slices.Contains(SupportedInitramfsImageTypes(), string(ft)) { + return fmt.Errorf("invalid initramfs image type (%s)", ft) + } + + return nil +} + +// SupportedImageFormatTypes returns all valid image format types. +func SupportedInitramfsImageTypes() []string { + return supportedInitramfsImageTypes +} diff --git a/toolkit/tools/imagecustomizerapi/iso.go b/toolkit/tools/imagecustomizerapi/iso.go index eeb2b95906..93f7a9a84a 100644 --- a/toolkit/tools/imagecustomizerapi/iso.go +++ b/toolkit/tools/imagecustomizerapi/iso.go @@ -11,6 +11,7 @@ import ( type Iso struct { KernelCommandLine KernelCommandLine `yaml:"kernelCommandLine" json:"kernelCommandLine,omitempty"` AdditionalFiles AdditionalFileList `yaml:"additionalFiles" json:"additionalFiles,omitempty"` + InitramfsType InitramfsImageType `yaml:"initramfsType" json:"initramfsType,omitempty"` } func (i *Iso) IsValid() error { @@ -24,5 +25,10 @@ func (i *Iso) IsValid() error { return fmt.Errorf("invalid additionalFiles:\n%w", err) } + err = i.InitramfsType.IsValid() + if err != nil { + return fmt.Errorf("invalid initramfs type:\n%w", err) + } + return nil } diff --git a/toolkit/tools/imagecustomizerapi/pxe.go b/toolkit/tools/imagecustomizerapi/pxe.go index 98a55f8159..a7897fc7cf 100644 --- a/toolkit/tools/imagecustomizerapi/pxe.go +++ b/toolkit/tools/imagecustomizerapi/pxe.go @@ -13,8 +13,11 @@ var PxeIsoDownloadProtocols = []string{"ftp://", "http://", "https://", "nfs://" // Iso defines how the generated iso media should be configured. type Pxe struct { - IsoImageBaseUrl string `yaml:"isoImageBaseUrl" json:"isoImageBaseUrl,omitempty"` - IsoImageFileUrl string `yaml:"isoImageFileUrl" json:"isoImageFileUrl,omitempty"` + KernelCommandLine KernelCommandLine `yaml:"kernelCommandLine" json:"kernelCommandLine,omitempty"` + AdditionalFiles AdditionalFileList `yaml:"additionalFiles" json:"additionalFiles,omitempty"` + InitramfsType InitramfsImageType `yaml:"initramfsType" json:"initramfsType,omitempty"` + BootstrapBaseUrl string `yaml:"bootstrapBaseUrl" json:"bootstrapBaseUrl,omitempty"` + BootstrapFileUrl string `yaml:"bootstrapFileUrl" json:"bootstrapFileUrl,omitempty"` } func IsValidPxeUrl(urlString string) error { @@ -42,16 +45,37 @@ func IsValidPxeUrl(urlString string) error { } func (p *Pxe) IsValid() error { - if p.IsoImageBaseUrl != "" && p.IsoImageFileUrl != "" { - return fmt.Errorf("cannot specify both 'isoImageBaseUrl' and 'isoImageFileUrl' at the same time.") + err := p.KernelCommandLine.IsValid() + if err != nil { + return fmt.Errorf("invalid kernelCommandLine: %w", err) + } + + err = p.AdditionalFiles.IsValid() + if err != nil { + return fmt.Errorf("invalid additionalFiles:\n%w", err) + } + + err = p.InitramfsType.IsValid() + if err != nil { + return fmt.Errorf("invalid initramfs type:\n%w", err) + } + + if p.InitramfsType == InitramfsImageTypeFullOS { + if p.BootstrapBaseUrl != "" || p.BootstrapFileUrl != "" { + return fmt.Errorf("cannot specify either 'bootstrapBaseUrl' and 'bootstrapFileUrl' when the initramfs type is set to '%s'.", InitramfsImageTypeFullOS) + } + } + + if p.BootstrapBaseUrl != "" && p.BootstrapFileUrl != "" { + return fmt.Errorf("cannot specify both 'bootstrapBaseUrl' and 'bootstrapFileUrl' at the same time.") } - err := IsValidPxeUrl(p.IsoImageBaseUrl) + err = IsValidPxeUrl(p.BootstrapBaseUrl) if err != nil { - return fmt.Errorf("invalid 'isoImageBaseUrl' field value (%s):\n%w", p.IsoImageBaseUrl, err) + return fmt.Errorf("invalid 'bootstrapBaseUrl' field value (%s):\n%w", p.BootstrapBaseUrl, err) } - err = IsValidPxeUrl(p.IsoImageFileUrl) + err = IsValidPxeUrl(p.BootstrapFileUrl) if err != nil { - return fmt.Errorf("invalid 'isoImageFileUrl' field value (%s):\n%w", p.IsoImageFileUrl, err) + return fmt.Errorf("invalid 'bootstrapFileUrl' field value (%s):\n%w", p.BootstrapFileUrl, err) } return nil } diff --git a/toolkit/tools/imagecustomizerapi/schema.json b/toolkit/tools/imagecustomizerapi/schema.json index 5e8b4b10b1..ec5212c52d 100644 --- a/toolkit/tools/imagecustomizerapi/schema.json +++ b/toolkit/tools/imagecustomizerapi/schema.json @@ -207,6 +207,9 @@ }, "additionalFiles": { "$ref": "#/$defs/AdditionalFileList" + }, + "initramfsType": { + "type": "string" } }, "additionalProperties": false, @@ -443,10 +446,19 @@ }, "Pxe": { "properties": { - "isoImageBaseUrl": { + "kernelCommandLine": { + "$ref": "#/$defs/KernelCommandLine" + }, + "additionalFiles": { + "$ref": "#/$defs/AdditionalFileList" + }, + "initramfsType": { + "type": "string" + }, + "bootstrapBaseUrl": { "type": "string" }, - "isoImageFileUrl": { + "bootstrapFileUrl": { "type": "string" } }, diff --git a/toolkit/tools/internal/initrdutils/initrdutils.go b/toolkit/tools/internal/initrdutils/initrdutils.go new file mode 100644 index 0000000000..b9919883fd --- /dev/null +++ b/toolkit/tools/internal/initrdutils/initrdutils.go @@ -0,0 +1,221 @@ +// Copyright Microsoft Corporation. +// Licensed under the MIT License. + +package initrdutils + +import ( + "fmt" + "io" + "os" + "path/filepath" + "syscall" + + "github.com/cavaliercoder/go-cpio" + "github.com/klauspost/pgzip" +) + +func CreateInitrdImageFromFolder(inputDir, outputInitrdImagePath string) (err error) { + // The `inputDir` permissions will become the `/` permissions when the initrd + // is mounted. This needs to be 0755 or some processes will fail to function + // correctly. + err = os.Chmod(inputDir, 0755) + if err != nil { + return fmt.Errorf("failed to change folder permissions for (%s):\n%w", inputDir, err) + } + + outputFile, err := os.Create(outputInitrdImagePath) + if err != nil { + return fmt.Errorf("failed to create image file (%s):\n%w", outputInitrdImagePath, err) + } + defer outputFile.Close() + + gzipWriter := pgzip.NewWriter(outputFile) + defer gzipWriter.Close() + + cpioWriter := cpio.NewWriter(gzipWriter) + defer func() { + closeErr := cpioWriter.Close() + if err != nil { + err = closeErr + } + }() + + // Traverse the directory structure and add all the files/directories/links to the archive. + err = filepath.Walk(inputDir, func(path string, info os.FileInfo, fileErr error) (err error) { + if fileErr != nil { + return fmt.Errorf("encountered a file walk error on path (%s):\n%w", path, fileErr) + } + err = addFileToCpioArchive(inputDir, path, info, cpioWriter) + if err != nil { + return fmt.Errorf("failed to add (%s) to archive (%s):\n%w", path, outputInitrdImagePath, err) + } + return nil + }) + + return nil +} + +func buildCpioHeader(inputDir, path string, info os.FileInfo, link string) (cpioHeader *cpio.Header, err error) { + // Convert the OS header into a CPIO header + cpioHeader, err = cpio.FileInfoHeader(info, link) + if err != nil { + return nil, fmt.Errorf("failed to convert OS file info into a cpio header for (%s)\n%w", path, err) + } + + // Convert full path to relative path + relPath, err := filepath.Rel(inputDir, path) + if err != nil { + return nil, fmt.Errorf("failed to get relative path of (%s) using root (%s):\n%w", path, inputDir, err) + } + cpioHeader.Name = relPath + + // Set owners (cpio.FileInfoHeader() does not set the owners) + stat, ok := info.Sys().(*syscall.Stat_t) + if !ok { + return nil, fmt.Errorf("failed to get file stat of (%s)", path) + } + cpioHeader.UID = int(stat.Uid) + cpioHeader.GID = int(stat.Gid) + + return cpioHeader, nil +} + +func addFileToCpioArchive(inputDir, path string, info os.FileInfo, cpioWriter *cpio.Writer) (err error) { + var link string + if info.Mode()&os.ModeSymlink != 0 { + link, err = os.Readlink(path) + if err != nil { + return fmt.Errorf("failed to read link information of (%s):\n%w", path, err) + } + } + + cpioHeader, err := buildCpioHeader(inputDir, path, info, link) + if err != nil { + return fmt.Errorf("failed to construct cpio file header for (%s)\n%w", path, err) + } + + err = cpioWriter.WriteHeader(cpioHeader) + if err != nil { + return fmt.Errorf("failed to write cpio header for (%s)\n%w", path, err) + } + + if info.Mode().IsRegular() { + fileToAdd, err := os.Open(path) + if err != nil { + return fmt.Errorf("failed to open (%s)\n%w", path, err) + } + defer fileToAdd.Close() + + _, err = io.Copy(cpioWriter, fileToAdd) + if err != nil { + return fmt.Errorf("failed to write (%s) to cpio archive\n%w", path, err) + } + } else { + if info.Mode()&os.ModeSymlink != 0 { + _, err = cpioWriter.Write([]byte(link)) + if err != nil { + return fmt.Errorf("failed to write link (%s)\n%w", path, err) + } + } + + // For all other special files, they will be of size 0 and only contain + // the header in the archive. + } + + return nil +} + +func updateFileOwnership(path string, fileMode os.FileMode, uid, gid int) (err error) { + err = os.Chown(path, uid, gid) + if err != nil { + return fmt.Errorf("failed to set ownership on extracted (%s) to (%d,%d):\n%w", path, uid, gid, err) + } + + // The golang implementation maps the setuid/setgid/sticky flags to bits + // different from those defined in native C implementation. As a result, + // we must convert between them. + // See https://github.com/golang/go/blob/release-branch.go1.24/src/os/stat_js.go + + if fileMode&cpio.ModePerm != 0 { + fileModeAdjusted := os.FileMode(fileMode).Perm() + if fileMode&syscall.S_ISUID != 0 { + fileModeAdjusted |= os.ModeSetuid + } + if fileMode&syscall.S_ISGID != 0 { + fileModeAdjusted |= os.ModeSetgid + } + if fileMode&syscall.S_ISVTX != 0 { + fileModeAdjusted |= os.ModeSticky + } + + err = os.Chmod(path, fileModeAdjusted) + if err != nil { + return fmt.Errorf("failed to change mode for file (%s) to (%#o):\n%w", path, fileMode, err) + } + } + return nil +} + +func CreateFolderFromInitrdImage(inputInitrdImagePath, outputDir string) (err error) { + inputInitrdImageFile, err := os.Open(inputInitrdImagePath) + if err != nil { + return fmt.Errorf("failed to open file (%s):\n%w", inputInitrdImagePath, err) + } + defer inputInitrdImageFile.Close() + + pgzipReader, err := pgzip.NewReader(inputInitrdImageFile) + if err != nil { + return fmt.Errorf("failed to create a pgzip reader for (%s):\n%w", inputInitrdImagePath, err) + } + defer pgzipReader.Close() + + cpioReader := cpio.NewReader(pgzipReader) + for { + cpioHeader, err := cpioReader.Next() + if err == io.EOF { + break + } + if err != nil { + return fmt.Errorf("failed to read cpio header from (%s):\n%w", inputInitrdImagePath, err) + } + + path := filepath.Join(outputDir, cpioHeader.Name) + fileMode := os.FileMode(cpioHeader.Mode & (cpio.ModePerm | cpio.ModeSetuid | cpio.ModeSetgid | cpio.ModeSticky)) + fileType := cpioHeader.Mode & cpio.ModeType + + switch fileType { + case cpio.ModeDir: + err := os.MkdirAll(path, fileMode) + if err != nil { + return fmt.Errorf("failed to create directory (%s):\n%w", path, err) + } + err = updateFileOwnership(path, fileMode, cpioHeader.UID, cpioHeader.GID) + if err != nil { + return fmt.Errorf("failed to update ownership of directory (%s)\n%w", path, err) + } + case cpio.ModeRegular: + destFile, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, fileMode) + if err != nil { + return fmt.Errorf("failed to create file (%s):\n%w", path, err) + } + _, err = io.Copy(destFile, cpioReader) + destFile.Close() + if err != nil { + return fmt.Errorf("failed to write file (%s):\n%w", path, err) + } + err = updateFileOwnership(path, fileMode, cpioHeader.UID, cpioHeader.GID) + if err != nil { + return fmt.Errorf("failed to update ownership of file (%s)\n%w", path, err) + } + case cpio.ModeSymlink: + err = os.Symlink(cpioHeader.Linkname, path) + if err != nil { + return fmt.Errorf("failed to create symbolic link (%s) to (%s)\n%w", cpioHeader.Linkname, path, err) + } + default: + return fmt.Errorf("unsupported type (%s) in cpio archive (%s)", fileType, inputInitrdImagePath) + } + } + + return nil +} diff --git a/toolkit/tools/internal/isogenerator/isogenerator.go b/toolkit/tools/internal/isogenerator/isogenerator.go index f7c0271fab..7b08a80b27 100644 --- a/toolkit/tools/internal/isogenerator/isogenerator.go +++ b/toolkit/tools/internal/isogenerator/isogenerator.go @@ -80,7 +80,7 @@ func GenerateIso(config IsoGenConfig) error { } func BuildIsoImage(stagingPath string, enableBiosBoot bool, isoOsFilesDirPath string, outputImagePath string) error { - logger.Log.Infof("Generating ISO image (%s) using (%s).", outputImagePath, stagingPath) + logger.Log.Infof("Creating ISO image (%s) using (%s).", outputImagePath, stagingPath) // For detailed parameter explanation see: https://linux.die.net/man/8/mkisofs. // Mkisofs requires all argument paths to be relative to the input directory. @@ -143,13 +143,13 @@ func copyInitrd(info isoGenInfo) error { } func BuildIsoBootImage(buildDir string, sourceShimPath string, sourceGrubPath string, outputImagePath string) (err error) { + logger.Log.Infof("Creating ISO bootloader image (%s)", outputImagePath) + const ( blockSizeInBytes = 1024 * 1024 numberOfBlocksToCopy = 3 ) - logger.Log.Info("Preparing ISO's bootloaders.") - ddArgs := []string{ "if=/dev/zero", // Zero device to read a stream of zeroed bytes from. fmt.Sprintf("of=%s", outputImagePath), // Output file. diff --git a/toolkit/tools/internal/tarutils/tarutils.go b/toolkit/tools/internal/tarutils/tarutils.go new file mode 100644 index 0000000000..446eb13a92 --- /dev/null +++ b/toolkit/tools/internal/tarutils/tarutils.go @@ -0,0 +1,140 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +package tarutils + +import ( + "archive/tar" + "compress/gzip" + "fmt" + "io" + "os" + "path/filepath" + "strings" + + "github.com/microsoft/azurelinux/toolkit/tools/internal/logger" +) + +func CreateTarGzArchive(sourceDir, outputArchivePath string) error { + logger.Log.Infof("Creating archive (%s) from (%s)", outputArchivePath, sourceDir) + + outFile, err := os.Create(outputArchivePath) + if err != nil { + return fmt.Errorf("failed to create archive (%s):\n%w", outputArchivePath, err) + } + defer outFile.Close() + + gw := gzip.NewWriter(outFile) + defer gw.Close() + + tw := tar.NewWriter(gw) + defer tw.Close() + + err = filepath.Walk(sourceDir, func(file string, info os.FileInfo, err error) error { + if err != nil { + return err + } + + header, err := tar.FileInfoHeader(info, info.Name()) + if err != nil { + return err + } + + // Adjust the header name to maintain folder structure + relPath, err := filepath.Rel(sourceDir, file) + if err != nil { + return err + } + header.Name = filepath.ToSlash(relPath) // Ensure forward slashes + + if err := tw.WriteHeader(header); err != nil { + return err + } + + // If it's a directory, nothing more to do + if info.IsDir() { + return nil + } + + // Write file contents + f, err := os.Open(file) + if err != nil { + return err + } + defer f.Close() + + _, err = io.Copy(tw, f) + return err + }) + + if err != nil { + return fmt.Errorf("failed to create archive (%s):\n%w", outputArchivePath, err) + } + + return nil +} + +func ExpandTarGzArchive(sourceArchivePath, outputDir string) error { + logger.Log.Infof("Expanding archive (%s) to (%s)", sourceArchivePath, outputDir) + + f, err := os.Open(sourceArchivePath) + if err != nil { + return fmt.Errorf("failed to archive (%s):\n%w", sourceArchivePath, err) + } + defer f.Close() + + gzr, err := gzip.NewReader(f) + if err != nil { + return fmt.Errorf("failed to create gzip reader for (%s):\n%w", sourceArchivePath, err) + } + defer gzr.Close() + + tr := tar.NewReader(gzr) + for { + header, err := tr.Next() + if err == io.EOF { + break + } + if err != nil { + return fmt.Errorf("failed to read header from archive:\n%w", err) + } + + // Ensure the name is not a directory traversal element (e.g. '..') or + // an absolute path. We call filepath.Clean() to normalize it before + // checking. + cleanName := filepath.Clean(header.Name) + if strings.Contains(cleanName, "..") || filepath.IsAbs(cleanName) { + return fmt.Errorf("unallowed file reference in archive. (%s) may reference a file outside the expansion root (%s)", header.Name, outputDir) + } + + target := filepath.Join(outputDir, cleanName) + + switch header.Typeflag { + case tar.TypeDir: + if err := os.MkdirAll(target, os.FileMode(header.Mode)); err != nil { + return fmt.Errorf("failed to create folder (%s)\n%w", target, err) + } + case tar.TypeReg: + if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil { + return fmt.Errorf("failed to create parent folder for (%s)\n%w", target, err) + } + outFile, err := os.Create(target) + if err != nil { + return fmt.Errorf("failed to create (%s):\n%w", target, err) + } + defer outFile.Close() + _, err = io.Copy(outFile, tr) + if err != nil { + return fmt.Errorf("failed to copy (%s) from archive:\n%w", target, err) + } + outFile.Close() + + if err := os.Chmod(target, os.FileMode(header.Mode)); err != nil { + return fmt.Errorf("failed to set permissions (%d) on (%s):\n%w", os.FileMode(header.Mode), target, err) + } + default: + return fmt.Errorf("failed to process unsupported file type in archive (%s): (%v)", target, header.Typeflag) + } + } + return nil +} diff --git a/toolkit/tools/pkg/imagecustomizerlib/artifactsinputoutput_test.go b/toolkit/tools/pkg/imagecustomizerlib/artifactsinputoutput_test.go index 1363f62f89..8092658532 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/artifactsinputoutput_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/artifactsinputoutput_test.go @@ -33,7 +33,7 @@ func TestOutputAndInjectArtifacts(t *testing.T) { // Customize image err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizebootloader_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizebootloader_test.go index fc735cbcf4..b1dc26d85a 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizebootloader_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizebootloader_test.go @@ -41,7 +41,7 @@ func testCustomizeImageMultiKernel(t *testing.T, testName string, imageType base // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizefiles_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizefiles_test.go index a95430928f..a941af2599 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizefiles_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizefiles_test.go @@ -80,7 +80,7 @@ func TestCustomizeImageAdditionalFiles(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -126,7 +126,7 @@ func TestCustomizeImageAdditionalFilesInfiniteFile(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "failed to copy (/dev/zero)") assert.ErrorContains(t, err, "no space left on device") } @@ -202,7 +202,7 @@ func TestCustomizeImageAdditionalDirs(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -258,7 +258,7 @@ func TestCustomizeImageAdditionalDirsInfiniteFile(t *testing.T) { // Customize image. err = CustomizeImage(buildDir, testTmpDir, &config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "failed to copy directory") assert.ErrorContains(t, err, "failed to copy file") assert.ErrorContains(t, err, "no space left on device") diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizehostname_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizehostname_test.go index aab3b620f9..920ba97da4 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizehostname_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizehostname_test.go @@ -48,7 +48,7 @@ func TestCustomizeImageHostname(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizeoverlays_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizeoverlays_test.go index ed29c0f74e..57705f6001 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizeoverlays_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizeoverlays_test.go @@ -24,7 +24,7 @@ func TestCustomizeImageOverlays(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -88,7 +88,7 @@ func TestCustomizeImageOverlaysSELinux(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizepackages_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizepackages_test.go index 7c9b33f9af..1a873ba132 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizepackages_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizepackages_test.go @@ -45,7 +45,7 @@ func TestCustomizeImagePackagesAddOfflineDir(t *testing.T) { } err = CustomizeImage(buildDir, testDir, &config, baseImage, []string{downloadedRpmsTmpDir}, outImageFilePath, - "raw", "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + "raw", false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -88,7 +88,7 @@ func TestCustomizeImagePackagesAddOfflineDir(t *testing.T) { } err = CustomizeImage(buildDir, testDir, &config, outImageFilePath, []string{downloadedRpmsTmpDir}, outImageFilePath, - "raw", "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + "raw", false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -157,7 +157,7 @@ func testCustomizeImagePackagesAddOfflineLocalRepoHelper(t *testing.T, testName // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, rpmSources, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -186,7 +186,7 @@ func TestCustomizeImagePackagesUpdate(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -222,7 +222,7 @@ func TestCustomizeImagePackagesDiskSpace(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "failed to customize raw image") assert.ErrorContains(t, err, "failed to install packages ([gcc])") } @@ -240,7 +240,7 @@ func TestCustomizeImagePackagesUrlSource(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, []string{repoFile}, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -270,7 +270,7 @@ func TestCustomizeImagePackagesBadRepo(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, []string{repoFile}, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "failed to refresh tdnf repo metadata") } @@ -327,7 +327,7 @@ func TestCustomizeImagePackagesSnapshotTime(t *testing.T) { } err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, - "raw", "", true, "") + "raw", true, "") if !assert.NoError(t, err) { return } @@ -375,7 +375,7 @@ func TestCustomizeImagePackagesCliSnapshotTimeOverridesConfigFile(t *testing.T) // Set the snapshot time in CLI to a date before jq-1.7.1-2 (2025-03-18) was published err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, - "raw", "", true, snapshotTimeCLI) + "raw", true, snapshotTimeCLI) if !assert.NoError(t, err) { return } @@ -418,7 +418,7 @@ func TestCustomizeImagePackagesSnapshotTimeWithoutPreviewFlagFails(t *testing.T) } err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, - "raw", "", true, "") + "raw", true, "") assert.ErrorContains(t, err, "snapshotTime") assert.ErrorContains(t, err, "preview feature") } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizepartitions_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizepartitions_test.go index 53eac4a4e8..bbe9a896a4 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizepartitions_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizepartitions_test.go @@ -48,7 +48,7 @@ func testCustomizeImagePartitionsToEfi(t *testing.T, testName string, imageType // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -136,7 +136,7 @@ func TestCustomizeImagePartitionsSizeOnly(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -231,7 +231,7 @@ func testCustomizeImagePartitionsToLegacy(t *testing.T, testName string, imageTy // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -283,7 +283,7 @@ func testCustomizeImageKernelCommandLineHelper(t *testing.T, testName string, ba // Customize image. err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -352,7 +352,7 @@ func testCustomizeImageNewUUIDsHelper(t *testing.T, testName string, imageType b // Customize image. err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizeselinux_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizeselinux_test.go index b971972339..4ca2817613 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizeselinux_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizeselinux_test.go @@ -35,7 +35,7 @@ func testCustomizeImageSELinuxHelper(t *testing.T, testName string, imageType ba // This tests enabling SELinux on a non-SELinux image. configFile := filepath.Join(testDir, "selinux-force-enforcing.yaml") err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -64,7 +64,7 @@ func testCustomizeImageSELinuxHelper(t *testing.T, testName string, imageType ba // This tests disabling (but not removing) SELinux on an SELinux enabled image. configFile = filepath.Join(testDir, "selinux-disabled.yaml") err = CustomizeImageWithConfigFile(buildDir, configFile, outImageFilePath, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -93,7 +93,7 @@ func testCustomizeImageSELinuxHelper(t *testing.T, testName string, imageType ba // This tests enabling SELinux on an image with SELinux installed but disabled. configFile = filepath.Join(testDir, "selinux-permissive.yaml") err = CustomizeImageWithConfigFile(buildDir, configFile, outImageFilePath, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -132,7 +132,7 @@ func testCustomizeImageSELinuxAndPartitionsHelper(t *testing.T, testName string, // This tests enabling SELinux on a non-SELinux image. configFile := filepath.Join(testDir, "partitions-selinux-enforcing.yaml") err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -181,7 +181,7 @@ func TestCustomizeImageSELinuxNoPolicy(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "SELinux is enabled but the (/etc/selinux/config) file is missing") assert.ErrorContains(t, err, "please ensure an SELinux policy is installed") assert.ErrorContains(t, err, "the 'selinux-policy' package provides the default policy") diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizeservices_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizeservices_test.go index 997b3b40d1..7cf1e5a379 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizeservices_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizeservices_test.go @@ -22,7 +22,7 @@ func TestCustomizeImageServicesEnableDisable(t *testing.T) { // Customize image. configFile := filepath.Join(testDir, "services-config.yaml") err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -63,7 +63,7 @@ func TestCustomizeImageServicesEnableUnknown(t *testing.T) { } err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "failed to enable service (chocolate-chip-muffin)") assert.ErrorContains(t, err, "chocolate-chip-muffin.service does not exist") } @@ -87,7 +87,7 @@ func TestCustomizeImageServicesDisableUnknown(t *testing.T) { } err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "failed to disable service (chocolate-chip-muffin)") assert.ErrorContains(t, err, "No such file or directory") } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizeuki_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizeuki_test.go index cfba19a5c7..1527d2a681 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizeuki_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizeuki_test.go @@ -30,7 +30,7 @@ func TestCustomizeImageVerityUsrUki(t *testing.T) { // Customize image. err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizeusers_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizeusers_test.go index 498650d631..3f0d901f03 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizeusers_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizeusers_test.go @@ -79,7 +79,7 @@ func TestCustomizeImageUsers(t *testing.T) { // Customize image. err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -163,7 +163,7 @@ func TestCustomizeImageUsersExitingUserHomeDir(t *testing.T) { // Customize image. err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "cannot set home directory (/home/root) on a user (root) that already exists") } @@ -187,7 +187,7 @@ func TestCustomizeImageUsersExitingUserUid(t *testing.T) { // Customize image. err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "cannot set UID (1) on a user (root) that already exists") } diff --git a/toolkit/tools/pkg/imagecustomizerlib/customizeverity_test.go b/toolkit/tools/pkg/imagecustomizerlib/customizeverity_test.go index e70f1aac85..ea7cde570e 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/customizeverity_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/customizeverity_test.go @@ -42,7 +42,7 @@ func testCustomizeImageVerityHelper(t *testing.T, testName string, imageType bas // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -51,7 +51,7 @@ func testCustomizeImageVerityHelper(t *testing.T, testName string, imageType bas // Recustomize the image. err = CustomizeImageWithConfigFile(buildDir, configFile, outImageFilePath, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -143,7 +143,7 @@ func testCustomizeImageVerityCosiExtractHelper(t *testing.T, testName string, im // Customize image, shrink partitions, and split the partitions into individual files. err = CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, "cosi", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return @@ -349,7 +349,7 @@ func testCustomizeImageVerityUsrHelper(t *testing.T, testName string, imageType // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -359,7 +359,7 @@ func testCustomizeImageVerityUsrHelper(t *testing.T, testName string, imageType // Recustomize image. // This helps verify that verity-enabled images can be recustomized. err = CustomizeImageWithConfigFile(buildDir, configFile, outImageFilePath, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -435,14 +435,14 @@ func testCustomizeImageVerityUsr2StageHelper(t *testing.T, testName string, imag // Stage 1: Create the partitions for verity. err := CustomizeImageWithConfigFile(buildDir, stage1ConfigFile, baseImage, nil, stage1FilePath, "qcow2", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } // Stage 2: Enable verity. err = CustomizeImageWithConfigFile(buildDir, stage2ConfigFile, stage1FilePath, nil, stage2FilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -474,7 +474,7 @@ func testCustomizeImageVerityReinitRootHelper(t *testing.T, testName string, ima // Stage 1: Initialize verity. err := CustomizeImageWithConfigFile(buildDir, stage1ConfigFile, baseImage, nil, stage1FilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -483,7 +483,7 @@ func testCustomizeImageVerityReinitRootHelper(t *testing.T, testName string, ima // Stage 2a: Reinitialize verity. err = CustomizeImageWithConfigFile(buildDir, stage2aConfigFile, stage1FilePath, nil, stage2FilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -492,7 +492,7 @@ func testCustomizeImageVerityReinitRootHelper(t *testing.T, testName string, ima // Stage 2b: Reinitialize verity + hard-reset bootloader. err = CustomizeImageWithConfigFile(buildDir, stage2bConfigFile, stage1FilePath, nil, stage2FilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -523,7 +523,7 @@ func testCustomizeImageVerityReinitUsrHelper(t *testing.T, testName string, imag // Stage 1: Initialize verity. err := CustomizeImageWithConfigFile(buildDir, stage1ConfigFile, baseImage, nil, stage1FilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -532,7 +532,7 @@ func testCustomizeImageVerityReinitUsrHelper(t *testing.T, testName string, imag // Stage 2: Reinitialize verity. err = CustomizeImageWithConfigFile(buildDir, stage2ConfigFile, stage1FilePath, nil, stage2FilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/extractpartitions_test.go b/toolkit/tools/pkg/imagecustomizerlib/extractpartitions_test.go index 9f90c68554..fad36bd077 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/extractpartitions_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/extractpartitions_test.go @@ -259,7 +259,7 @@ func TestCustomizeImageNopShrink(t *testing.T) { outImageFilePath := filepath.Join(testTempDir, "image.cosi") // Customize image. - err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "cosi", "" /*outputPXEArtifactsDir*/, true, "" /*packageSnapshotTime*/) + err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "cosi", true, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -352,7 +352,7 @@ func TestCustomizeImageExtractEmptyPartition(t *testing.T) { outImageFilePath := filepath.Join(testTempDir, "image.raw") // Customize image. - err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "cosi", "", false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "cosi", false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/imagecustomizer.go b/toolkit/tools/pkg/imagecustomizerlib/imagecustomizer.go index 4c91923db7..997a07727f 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/imagecustomizer.go +++ b/toolkit/tools/pkg/imagecustomizerlib/imagecustomizer.go @@ -69,12 +69,12 @@ type ImageCustomizerParameters struct { rawImageFile string // output image - outputImageFormat imagecustomizerapi.ImageFormatType - outputIsIso bool - outputImageFile string - outputImageDir string - outputImageBase string - outputPXEArtifactsDir string + outputImageFormat imagecustomizerapi.ImageFormatType + outputIsIso bool + outputIsPxe bool + outputImageFile string + outputImageDir string + outputImageBase string imageUuid [UuidSize]byte imageUuidStr string @@ -100,7 +100,7 @@ func createImageCustomizerParameters(buildDir string, inputImageFile string, configPath string, config *imagecustomizerapi.Config, useBaseImageRpmRepos bool, rpmsSources []string, - outputImageFormat string, outputImageFile string, outputPXEArtifactsDir string, packageSnapshotTime string, + outputImageFormat string, outputImageFile string, packageSnapshotTime string, ) (*ImageCustomizerParameters, error) { ic := &ImageCustomizerParameters{} @@ -164,20 +164,15 @@ func createImageCustomizerParameters(buildDir string, ic.outputImageBase = strings.TrimSuffix(filepath.Base(ic.outputImageFile), filepath.Ext(ic.outputImageFile)) ic.outputImageDir = filepath.Dir(ic.outputImageFile) - ic.outputPXEArtifactsDir = outputPXEArtifactsDir ic.outputIsIso = ic.outputImageFormat == imagecustomizerapi.ImageFormatTypeIso - ic.packageSnapshotTime = packageSnapshotTime - - if ic.outputPXEArtifactsDir != "" && !ic.outputIsIso { - return nil, fmt.Errorf("the output PXE artifacts directory ('--output-pxe-artifacts-dir') can be specified only if the output format is an iso image.") - } + ic.outputIsPxe = ic.outputImageFormat == imagecustomizerapi.ImageFormatTypePxe if ic.inputIsIso { // While re-creating a disk image from the iso is technically possible, // we are choosing to not implement it until there is a need. - if !ic.outputIsIso { - return nil, fmt.Errorf("generating a non-iso image from an iso image is not supported") + if !ic.outputIsIso && !ic.outputIsPxe { + return nil, fmt.Errorf("cannot generate output format (%s) from the given input format (%s)", ic.outputImageFormat, ic.inputImageFormat) } // While defining a storage configuration can work when the input image is @@ -188,12 +183,14 @@ func createImageCustomizerParameters(buildDir string, } } + ic.packageSnapshotTime = packageSnapshotTime + return ic, nil } func CustomizeImageWithConfigFile(buildDir string, configFile string, inputImageFile string, rpmsSources []string, outputImageFile string, outputImageFormat string, - outputPXEArtifactsDir string, useBaseImageRpmRepos bool, packageSnapshotTime string, + useBaseImageRpmRepos bool, packageSnapshotTime string, ) error { var err error @@ -212,7 +209,7 @@ func CustomizeImageWithConfigFile(buildDir string, configFile string, inputImage } err = CustomizeImage(buildDir, absBaseConfigPath, &config, inputImageFile, rpmsSources, outputImageFile, outputImageFormat, - outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + useBaseImageRpmRepos, packageSnapshotTime) if err != nil { return err } @@ -231,7 +228,7 @@ func cleanUp(ic *ImageCustomizerParameters) error { func CustomizeImage(buildDir string, baseConfigPath string, config *imagecustomizerapi.Config, inputImageFile string, rpmsSources []string, outputImageFile string, outputImageFormat string, - outputPXEArtifactsDir string, useBaseImageRpmRepos bool, packageSnapshotTime string, + useBaseImageRpmRepos bool, packageSnapshotTime string, ) error { _, span := otel.GetTracerProvider().Tracer(OtelTracerName).Start(context.Background(), "CustomizeImage") span.SetAttributes( @@ -246,7 +243,7 @@ func CustomizeImage(buildDir string, baseConfigPath string, config *imagecustomi imageCustomizerParameters, err := createImageCustomizerParameters(buildDir, inputImageFile, baseConfigPath, config, useBaseImageRpmRepos, rpmsSources, - outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + outputImageFormat, outputImageFile, packageSnapshotTime) if err != nil { return fmt.Errorf("invalid parameters:\n%w", err) } @@ -322,7 +319,7 @@ func CustomizeImage(buildDir string, baseConfigPath string, config *imagecustomi } func convertInputImageToWriteableFormat(ic *ImageCustomizerParameters) (*IsoArtifactsStore, error) { - logger.Log.Infof("Converting input image to a writeable format") + logger.Log.Infof("Converting input image to a writeable format image (%s)", ic.rawImageFile) if ic.inputIsIso { @@ -331,12 +328,21 @@ func convertInputImageToWriteableFormat(ic *ImageCustomizerParameters) (*IsoArti return inputIsoArtifacts, fmt.Errorf("failed to create artifacts store from (%s):\n%w", ic.inputImageFile, err) } + rebuildFullOS := false + if ic.outputIsIso && ic.config.Iso != nil { + rebuildFullOS = (inputIsoArtifacts.files.squashfsImagePath == "" && ic.config.Iso.InitramfsType == imagecustomizerapi.InitramfsImageTypeBootstrap) || + (inputIsoArtifacts.files.squashfsImagePath != "" && ic.config.Iso.InitramfsType == imagecustomizerapi.InitramfsImageTypeFullOS) + } else if ic.outputIsPxe && ic.config.Pxe != nil { + rebuildFullOS = (inputIsoArtifacts.files.squashfsImagePath == "" && ic.config.Pxe.InitramfsType == imagecustomizerapi.InitramfsImageTypeBootstrap) || + (inputIsoArtifacts.files.squashfsImagePath != "" && ic.config.Pxe.InitramfsType == imagecustomizerapi.InitramfsImageTypeFullOS) + } + // If the input is a LiveOS iso and there are OS customizations // defined, we create a writeable disk image so that mic can modify // it. If no OS customizations are defined, we can skip this step and // just re-use the existing squashfs. - if ic.customizeOSPartitions { - err = createWriteableImageFromArtifacts(ic.buildDirAbs, inputIsoArtifacts.files, ic.rawImageFile) + if ic.customizeOSPartitions || rebuildFullOS { + err = createWriteableImageFromArtifacts(ic.buildDirAbs, inputIsoArtifacts, ic.rawImageFile) if err != nil { return nil, fmt.Errorf("failed to create writeable image:\n%w", err) } @@ -433,6 +439,8 @@ func customizeOSContents(ic *ImageCustomizerParameters) error { return nil } + logger.Log.Infof("Customizing OS Contents (%s)", ic.rawImageFile) + // The code beyond this point assumes the OS object is always present. To // change the code to check before every usage whether the OS object is // present or not will lead to a messy mix of if statements that do not @@ -533,20 +541,37 @@ func convertWriteableFormatToOutputImage(ic *ImageCustomizerParameters, inputIso return err } - case imagecustomizerapi.ImageFormatTypeIso: + case imagecustomizerapi.ImageFormatTypeIso, imagecustomizerapi.ImageFormatTypePxe: + rebuildFullOsImage := false + + // Decide whether we need to re-build the full OS image or not if ic.customizeOSPartitions || inputIsoArtifacts == nil { + rebuildFullOsImage = true + } else if inputIsoArtifacts != nil { + // Let's check if use is converting from full os initramfs to bootstrap initramfs + liveosConfig, err := buildLiveOSConfig(ic.outputImageFormat, ic.config.Iso, ic.config.Pxe) + if err != nil { + return fmt.Errorf("failed to build Live OS configuration\n%w", err) + } + + rebuildFullOsImage = (inputIsoArtifacts.files.squashfsImagePath == "" && liveosConfig.initramfsType == imagecustomizerapi.InitramfsImageTypeBootstrap) || + (inputIsoArtifacts.files.squashfsImagePath != "" && liveosConfig.initramfsType == imagecustomizerapi.InitramfsImageTypeFullOS) + } + + // Either re-build the full OS image, or just re-package the existing one + if rebuildFullOsImage { requestedSELinuxMode := imagecustomizerapi.SELinuxModeDefault if ic.config.OS != nil { requestedSELinuxMode = ic.config.OS.SELinux.Mode } - err := createLiveOSIsoImage(ic.buildDirAbs, ic.configPath, inputIsoArtifacts, requestedSELinuxMode, ic.config.Iso, ic.config.Pxe, - ic.rawImageFile, ic.outputImageFile, ic.outputPXEArtifactsDir) + err := createLiveOSFromRaw(ic.buildDirAbs, ic.configPath, inputIsoArtifacts, requestedSELinuxMode, + ic.config.Iso, ic.config.Pxe, ic.rawImageFile, ic.outputImageFormat, ic.outputImageFile) if err != nil { return fmt.Errorf("failed to create LiveOS iso image:\n%w", err) } } else { - err := createImageFromUnchangedOS(ic.buildDirAbs, ic.configPath, ic.config.Iso, ic.config.Pxe, - inputIsoArtifacts, ic.outputImageFile, ic.outputPXEArtifactsDir) + err := repackageLiveOS(ic.buildDirAbs, ic.configPath, ic.config.Iso, ic.config.Pxe, + inputIsoArtifacts, ic.outputImageFormat, ic.outputImageFile) if err != nil { return fmt.Errorf("failed to create LiveOS iso image:\n%w", err) } @@ -804,18 +829,21 @@ func validateOutput(baseConfigPath string, output imagecustomizerapi.Output, out return fmt.Errorf("output image file must be specified, either via the command line option '--output-image-file' or in the config file property 'output.image.path'") } - if outputImageFile != "" { - if isDir, err := file.DirExists(outputImageFile); err != nil { - return fmt.Errorf("invalid command-line option '--output-image-file': '%s'\n%w", outputImageFile, err) - } else if isDir { - return fmt.Errorf("invalid command-line option '--output-image-file': '%s'\nis a directory", outputImageFile) - } - } else { - outputImageAbsPath := file.GetAbsPathWithBase(baseConfigPath, output.Image.Path) - if isDir, err := file.DirExists(outputImageAbsPath); err != nil { - return fmt.Errorf("invalid config file property 'output.image.path': '%s'\n%w", output.Image.Path, err) - } else if isDir { - return fmt.Errorf("invalid config file property 'output.image.path': '%s'\nis a directory", output.Image.Path) + // Pxe output format allows the output to be a path. + if output.Image.Format != imagecustomizerapi.ImageFormatTypePxe { + if outputImageFile != "" { + if isDir, err := file.DirExists(outputImageFile); err != nil { + return fmt.Errorf("invalid command-line option '--output-image-file': '%s'\n%w", outputImageFile, err) + } else if isDir { + return fmt.Errorf("invalid command-line option '--output-image-file': '%s'\nis a directory", outputImageFile) + } + } else { + outputImageAbsPath := file.GetAbsPathWithBase(baseConfigPath, output.Image.Path) + if isDir, err := file.DirExists(outputImageAbsPath); err != nil { + return fmt.Errorf("invalid config file property 'output.image.path': '%s'\n%w", output.Image.Path, err) + } else if isDir { + return fmt.Errorf("invalid config file property 'output.image.path': '%s'\nis a directory", output.Image.Path) + } } } diff --git a/toolkit/tools/pkg/imagecustomizerlib/imagecustomizer_test.go b/toolkit/tools/pkg/imagecustomizerlib/imagecustomizer_test.go index 763659361c..b873642623 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/imagecustomizer_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/imagecustomizer_test.go @@ -58,7 +58,7 @@ func TestCustomizeImageEmptyConfig(t *testing.T) { // Customize image. err = CustomizeImage(buildDir, buildDir, &imagecustomizerapi.Config{}, baseImage, nil, outImageFilePath, "vhd", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -80,7 +80,7 @@ func TestCustomizeImageVhd(t *testing.T) { // Customize image to vhd. err := CustomizeImageWithConfigFile(buildDir, partitionsConfigFile, baseImage, nil, vhdImageFilePath, - "vhd", "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + "vhd", false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -96,7 +96,7 @@ func TestCustomizeImageVhd(t *testing.T) { // Customize image to vhd-fixed. err = CustomizeImageWithConfigFile(buildDir, noChangeConfigFile, vhdImageFilePath, nil, vhdFixedImageFilePath, - "vhd-fixed", "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + "vhd-fixed", false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -114,7 +114,7 @@ func TestCustomizeImageVhd(t *testing.T) { // Customize image to vhdx. err = CustomizeImageWithConfigFile(buildDir, noChangeConfigFile, vhdFixedImageFilePath, nil, vhdxImageFilePath, - "vhdx", "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + "vhdx", false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -538,13 +538,12 @@ func TestCustomizeImage_InputImageFileSelection(t *testing.T) { rpmSources := []string{} outputImageFile := filepath.Join(buildDir, "image.vhd") outputImageFormat := filepath.Ext(outputImageFile)[1:] - outputPXEArtifactsDir := "" useBaseImageRpmRepos := false packageSnapshotTime := "" // Pass the input image file only through the argument. err := CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFile) err = os.Remove(outputImageFile) @@ -555,7 +554,7 @@ func TestCustomizeImage_InputImageFileSelection(t *testing.T) { // Pass the input image file only through the config. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFile) err = os.Remove(outputImageFile) @@ -567,7 +566,7 @@ func TestCustomizeImage_InputImageFileSelection(t *testing.T) { // Pass the input image file through both the config and the argument. The config's Path is ignored, so even though // it doesn't exist, there will be no error. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFile) err = os.Remove(outputImageFile) @@ -598,14 +597,13 @@ func TestCustomizeImage_InputImageFileAsRelativePath(t *testing.T) { rpmSources := []string{} outputImageFile := filepath.Join(buildDir, "image.vhd") outputImageFormat := filepath.Ext(outputImageFile)[1:] - outputPXEArtifactsDir := "" useBaseImageRpmRepos := false packageSnapshotTime := "" // Pass the input image file relative to the current working directory through the argument. This works because // paths on the command-line are expected to be relative to the current working directory. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFile) err = os.Remove(outputImageFile) @@ -615,7 +613,7 @@ func TestCustomizeImage_InputImageFileAsRelativePath(t *testing.T) { // The same as above but for the fake path. This fails because the file does not exist. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.Error(t, err) assert.ErrorContains(t, err, "doesnotexist.xxx: no such file or directory") assert.NoFileExists(t, outputImageFile) @@ -626,7 +624,7 @@ func TestCustomizeImage_InputImageFileAsRelativePath(t *testing.T) { // Pass the input image file relative to the config file through the config. This works because paths in the config // as expected to be relative to the config file. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFile) err = os.Remove(outputImageFile) @@ -636,7 +634,7 @@ func TestCustomizeImage_InputImageFileAsRelativePath(t *testing.T) { // The same as above but for the fake path. This fails because the file does not exist. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.Error(t, err) assert.ErrorContains(t, err, "doesnotexist.xxx: no such file or directory") } @@ -659,7 +657,7 @@ func TestCustomizeImageKernelCommandLineAdd(t *testing.T) { } err = CustomizeImage(buildDir, buildDir, config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -701,13 +699,12 @@ func TestCustomizeImage_OutputImageFileSelection(t *testing.T) { outputImageFile := outputImageFileAsArgument outputImageFormat := filepath.Ext(outputImageFile)[1:] - outputPXEArtifactsDir := "" useBaseImageRpmRepos := false packageSnapshotTime := "" // Pass the output image file only through the argument. err := CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFileAsArgument) err = os.Remove(outputImageFileAsArgument) @@ -718,7 +715,7 @@ func TestCustomizeImage_OutputImageFileSelection(t *testing.T) { // Pass the output image file only through the config. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, "", - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFilePathAsConfig) err = os.Remove(outputImageFilePathAsConfig) @@ -730,7 +727,7 @@ func TestCustomizeImage_OutputImageFileSelection(t *testing.T) { // Pass the output image file through both the config and the argument. The config's Path is ignored, so even though // it is a directory, there will be no error. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFileAsArgument) assert.NoFileExists(t, outputImageFilePathAsConfig) @@ -757,14 +754,13 @@ func TestCustomizeImage_OutputImageFileAsRelativePath(t *testing.T) { outputImageFile := outputImageFileRelativeToCwd outputImageFormat := filepath.Ext(outputImageFile)[1:] - outputPXEArtifactsDir := "" useBaseImageRpmRepos := false packageSnapshotTime := "" // Pass the output image file relative to the current working directory through the argument. This will create // the file at the absolute path. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, "" /*packageSnapshotTime*/) + outputImageFormat, useBaseImageRpmRepos, "" /*packageSnapshotTime*/) assert.NoError(t, err) assert.FileExists(t, outputImageFileAbsolute) err = os.Remove(outputImageFileAbsolute) @@ -776,7 +772,7 @@ func TestCustomizeImage_OutputImageFileAsRelativePath(t *testing.T) { // Pass the output image file relative to the config file through the config. This will create the file at the // absolute path. err = CustomizeImage(buildDir, baseConfigPath, config, inputImageFile, rpmSources, outputImageFile, - outputImageFormat, outputPXEArtifactsDir, useBaseImageRpmRepos, packageSnapshotTime) + outputImageFormat, useBaseImageRpmRepos, packageSnapshotTime) assert.NoError(t, err) assert.FileExists(t, outputImageFileAbsolute) err = os.Remove(outputImageFileAbsolute) @@ -801,7 +797,7 @@ func TestCustomizeImage_OutputImageFormatSelection(t *testing.T) { }, } err := CustomizeImage(buildDir, buildDir, config, baseImage, nil, "", "", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.NoError(t, err) assert.FileExists(t, outputImageFile) checkFileType(t, outputImageFile, outputImageFormatAsConfig) @@ -813,7 +809,7 @@ func TestCustomizeImage_OutputImageFormatSelection(t *testing.T) { // Pass the output image format only through the argument. config.Output.Image.Format = imagecustomizerapi.ImageFormatTypeNone err = CustomizeImage(buildDir, buildDir, config, baseImage, nil, "", outputImageFormatAsArg, - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.NoError(t, err) assert.FileExists(t, outputImageFile) checkFileType(t, outputImageFile, outputImageFormatAsArg) @@ -825,7 +821,7 @@ func TestCustomizeImage_OutputImageFormatSelection(t *testing.T) { // Pass the output image format through both the config and the argument. config.Output.Image.Format = imagecustomizerapi.ImageFormatType(outputImageFormatAsConfig) err = CustomizeImage(buildDir, buildDir, config, baseImage, nil, "", outputImageFormatAsArg, - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.NoError(t, err) assert.FileExists(t, outputImageFile) checkFileType(t, outputImageFile, outputImageFormatAsArg) @@ -863,12 +859,11 @@ func TestCreateImageCustomizerParameters_InputImageFileSelection(t *testing.T) { rpmsSources := []string{} outputImageFormat := "vhdx" outputImageFile := "out/image.vhdx" - outputPXEArtifactsDir := "" packageSnapshotTime := "" // The input image file should be set to the value in the config. ic, err := createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.inputImageFile, inputImageFileAsConfig) assert.Equal(t, ic.inputImageFormat, "vhdx") @@ -880,7 +875,7 @@ func TestCreateImageCustomizerParameters_InputImageFileSelection(t *testing.T) { // The input image file should be set to the value passed as an argument. ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.inputImageFile, inputImageFileAsArg) assert.Equal(t, ic.inputImageFormat, "vhdx") @@ -891,7 +886,7 @@ func TestCreateImageCustomizerParameters_InputImageFileSelection(t *testing.T) { // The input image file should be set to the value passed as an argument. ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.inputImageFile, inputImageFileAsArg) assert.Equal(t, ic.inputImageFormat, "vhdx") @@ -902,7 +897,7 @@ func TestCreateImageCustomizerParameters_InputImageFileSelection(t *testing.T) { outputImageFormat = "iso" outputImageFile = "out/image.iso" ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.inputImageFile, inputImageFileIsoAsArg) assert.Equal(t, ic.inputImageFormat, "iso") @@ -927,12 +922,11 @@ func TestCreateImageCustomizerParameters_OutputImageFileSelection(t *testing.T) rpmsSources := []string{} outputImageFormat := "vhd" outputImageFile := "" - outputPXEArtifactsDir := "" packageSnapshotTime := "" // The output image file is not specified in the config or as an argument, so the output image file will be empty. ic, err := createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.outputImageFile, "") @@ -941,7 +935,7 @@ func TestCreateImageCustomizerParameters_OutputImageFileSelection(t *testing.T) // The output image file should be set to the value in the config. ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.outputImageFile, outputImageFilePathAsConfig) assert.Equal(t, ic.outputImageBase, "image-as-config") @@ -953,7 +947,7 @@ func TestCreateImageCustomizerParameters_OutputImageFileSelection(t *testing.T) // The output image file should be set to the value passed as an argument. ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.outputImageFile, outputImageFilePathAsArg) assert.Equal(t, ic.outputImageBase, "image-as-arg") @@ -965,7 +959,7 @@ func TestCreateImageCustomizerParameters_OutputImageFileSelection(t *testing.T) // The output image file should be set to the value passed as an // argument. ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.outputImageFile, outputImageFilePathAsArg) assert.Equal(t, ic.outputImageBase, "image-as-arg") @@ -990,13 +984,12 @@ func TestCreateImageCustomizerParameters_OutputImageFormatSelection(t *testing.T rpmsSources := []string{} outputImageFormat := "" outputImageFile := filepath.Join(buildDir, "image.vhd") - outputPXEArtifactsDir := "" packageSnapshotTime := "" // The output image format is not specified in the config or as an // argument, so the output image format will be empty. ic, err := createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.outputImageFormat, imagecustomizerapi.ImageFormatTypeNone) @@ -1005,7 +998,7 @@ func TestCreateImageCustomizerParameters_OutputImageFormatSelection(t *testing.T // The output image file should be set to the value in the config. ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.outputImageFormat, imagecustomizerapi.ImageFormatType(outputImageFormatAsConfig)) @@ -1016,7 +1009,7 @@ func TestCreateImageCustomizerParameters_OutputImageFormatSelection(t *testing.T // The output image file should be set to the value passed as an // argument. ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.outputImageFormat, imagecustomizerapi.ImageFormatType(outputImageFormatAsArg)) @@ -1026,7 +1019,7 @@ func TestCreateImageCustomizerParameters_OutputImageFormatSelection(t *testing.T // The output image file should be set to the value passed as an // argument. ic, err = createImageCustomizerParameters(buildDir, inputImageFile, configPath, config, useBaseImageRpmRepos, - rpmsSources, outputImageFormat, outputImageFile, outputPXEArtifactsDir, packageSnapshotTime) + rpmsSources, outputImageFormat, outputImageFile, packageSnapshotTime) assert.NoError(t, err) assert.Equal(t, ic.outputImageFormat, imagecustomizerapi.ImageFormatType(outputImageFormatAsArg)) } diff --git a/toolkit/tools/pkg/imagecustomizerlib/installedkernelcheck_test.go b/toolkit/tools/pkg/imagecustomizerlib/installedkernelcheck_test.go index 2a526b0702..43283d4f6b 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/installedkernelcheck_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/installedkernelcheck_test.go @@ -20,6 +20,6 @@ func TestCustomizeImageMissingKernel(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.ErrorContains(t, err, "no installed kernel found") } diff --git a/toolkit/tools/pkg/imagecustomizerlib/kernelmoduleutils_test.go b/toolkit/tools/pkg/imagecustomizerlib/kernelmoduleutils_test.go index 3e40062ebb..5d4c1092c9 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/kernelmoduleutils_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/kernelmoduleutils_test.go @@ -228,7 +228,7 @@ func TestCustomizeImageKernelModules(t *testing.T) { // Customize image. err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/liveosisoartifactstore.go b/toolkit/tools/pkg/imagecustomizerlib/liveosisoartifactstore.go index 37ad5f8e3c..584798dd4f 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/liveosisoartifactstore.go +++ b/toolkit/tools/pkg/imagecustomizerlib/liveosisoartifactstore.go @@ -42,7 +42,6 @@ type IsoFilesStore struct { grubEfiPath string isoBootImagePath string isoGrubCfgPath string - pxeGrubCfgPath string savedConfigsFilePath string vmlinuzPath string initrdImagePath string @@ -234,8 +233,6 @@ func createIsoFilesStoreFromMountedImage(inputArtifactsStore *IsoArtifactsStore, targetPath = filepath.Join(filesStore.artifactsDir, "EFI/BOOT", isoGrubCfg) } filesStore.isoGrubCfgPath = targetPath - // We will place the pxe grub config next to the iso grub config. - filesStore.pxeGrubCfgPath = filepath.Join(filepath.Dir(filesStore.isoGrubCfgPath), pxeGrubCfg) scheduleAdditionalFile = false } @@ -415,8 +412,6 @@ func createIsoFilesStoreFromIsoImage(isoImageFile, storeDir string) (filesStore scheduleAdditionalFile = false case isoGrubCfgPath: filesStore.isoGrubCfgPath = isoFile - // We will place the pxe grub config next to the iso grub config. - filesStore.pxeGrubCfgPath = filepath.Join(filepath.Dir(filesStore.isoGrubCfgPath), pxeGrubCfg) scheduleAdditionalFile = false case liveOSImagePath: filesStore.squashfsImagePath = isoFile @@ -457,6 +452,7 @@ func createIsoInfoStoreFromIsoImage(savedConfigFile string) (infoStore *IsoInfoS // since we will not expand the rootfs and inspect its contents to get // such information. infoStore = &IsoInfoStore{ + kernelVersion: savedConfigs.OS.KernelVersion, dracutPackageInfo: savedConfigs.OS.DracutPackageInfo, selinuxPolicyPackageInfo: savedConfigs.OS.SELinuxPolicyPackageInfo, } @@ -465,7 +461,7 @@ func createIsoInfoStoreFromIsoImage(savedConfigFile string) (infoStore *IsoInfoS } func createIsoArtifactStoreFromMountedImage(inputArtifactsStore *IsoArtifactsStore, imageRootDir string, storeDir string) (artifactStore *IsoArtifactsStore, err error) { - logger.Log.Debugf("Creating ISO store (%s)", storeDir) + logger.Log.Infof("Creating ISO store (%s) from (%s)", storeDir, imageRootDir) err = os.MkdirAll(storeDir, os.ModePerm) if err != nil { @@ -490,7 +486,7 @@ func createIsoArtifactStoreFromMountedImage(inputArtifactsStore *IsoArtifactsSto } func createIsoArtifactStoreFromIsoImage(isoImageFile, storeDir string) (artifactStore *IsoArtifactsStore, err error) { - logger.Log.Debugf("Creating ISO store (%s)", storeDir) + logger.Log.Infof("Creating ISO store (%s) from (%s)", storeDir, isoImageFile) err = os.MkdirAll(storeDir, os.ModePerm) if err != nil { @@ -536,7 +532,6 @@ func dumpFilesStore(filesStore *IsoFilesStore) { logger.Log.Debugf("-- grubEfiPath = %s", fileExistsToString(filesStore.grubEfiPath)) logger.Log.Debugf("-- isoBootImagePath = %s", fileExistsToString(filesStore.isoBootImagePath)) logger.Log.Debugf("-- isoGrubCfgPath = %s", fileExistsToString(filesStore.isoGrubCfgPath)) - logger.Log.Debugf("-- pxeGrubCfgPath = %s", fileExistsToString(filesStore.pxeGrubCfgPath)) logger.Log.Debugf("-- savedConfigsFilePath = %s", fileExistsToString(filesStore.savedConfigsFilePath)) logger.Log.Debugf("-- vmlinuzPath = %s", fileExistsToString(filesStore.vmlinuzPath)) logger.Log.Debugf("-- initrdImagePath = %s", fileExistsToString(filesStore.initrdImagePath)) diff --git a/toolkit/tools/pkg/imagecustomizerlib/liveosisobuilder.go b/toolkit/tools/pkg/imagecustomizerlib/liveosisobuilder.go index 9795f429f7..ce063d1275 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/liveosisobuilder.go +++ b/toolkit/tools/pkg/imagecustomizerlib/liveosisobuilder.go @@ -9,18 +9,65 @@ import ( "path/filepath" "github.com/microsoft/azurelinux/toolkit/tools/imagecustomizerapi" - "github.com/microsoft/azurelinux/toolkit/tools/internal/file" "github.com/microsoft/azurelinux/toolkit/tools/internal/isogenerator" "github.com/microsoft/azurelinux/toolkit/tools/internal/logger" ) +const ( + defaultIsoImageName = "image.iso" +) + +type LiveOSConfig struct { + isPxe bool + kernelCommandLine imagecustomizerapi.KernelCommandLine + additionalFiles imagecustomizerapi.AdditionalFileList + initramfsType imagecustomizerapi.InitramfsImageType + bootstrapBaseUrl string + bootstrapFileUrl string +} + +func buildLiveOSConfig(outputFormat imagecustomizerapi.ImageFormatType, isoConfig *imagecustomizerapi.Iso, pxeConfig *imagecustomizerapi.Pxe) ( + config LiveOSConfig, err error) { + + switch outputFormat { + case imagecustomizerapi.ImageFormatTypeIso: + config.isPxe = false + if isoConfig != nil { + config.kernelCommandLine = isoConfig.KernelCommandLine + config.additionalFiles = isoConfig.AdditionalFiles + config.initramfsType = isoConfig.InitramfsType + } + // Set default initramfs type + if config.initramfsType == imagecustomizerapi.InitramfsImageTypeUnspecified { + config.initramfsType = imagecustomizerapi.InitramfsImageTypeBootstrap + } + case imagecustomizerapi.ImageFormatTypePxe: + config.isPxe = true + if pxeConfig != nil { + config.kernelCommandLine = pxeConfig.KernelCommandLine + config.additionalFiles = pxeConfig.AdditionalFiles + config.initramfsType = pxeConfig.InitramfsType + config.bootstrapBaseUrl = pxeConfig.BootstrapBaseUrl + config.bootstrapFileUrl = pxeConfig.BootstrapFileUrl + } + // Set default initramfs type + if config.initramfsType == imagecustomizerapi.InitramfsImageTypeUnspecified { + config.initramfsType = imagecustomizerapi.InitramfsImageTypeFullOS + } + default: + return config, fmt.Errorf("unsupported liveos output format (%s)", outputFormat) + } + + return config, nil +} + func populateWriteableRootfsDir(sourceDir, writeableRootfsDir string) error { - logger.Log.Debugf("Creating writeable rootfs") + logger.Log.Infof("Creating writeable rootfs (%s) from (%s)", writeableRootfsDir, sourceDir) err := os.MkdirAll(writeableRootfsDir, os.ModePerm) if err != nil { - return fmt.Errorf("failed to create folder %s:\n%w", writeableRootfsDir, err) + return fmt.Errorf("failed to create folder (%s):\n%w", writeableRootfsDir, err) } err = copyPartitionFiles(sourceDir+"/.", writeableRootfsDir) @@ -31,27 +78,47 @@ func populateWriteableRootfsDir(sourceDir, writeableRootfsDir string) error { return nil } -func createLiveOSIsoImage(buildDir, baseConfigPath string, inputArtifactsStore *IsoArtifactsStore, requestedSelinuxMode imagecustomizerapi.SELinuxMode, - isoConfig *imagecustomizerapi.Iso, pxeConfig *imagecustomizerapi.Pxe, rawImageFile, outputImagePath string, - outputPXEArtifactsDir string) (err error) { +func createLiveOSFromRaw(buildDir, baseConfigPath string, inputArtifactsStore *IsoArtifactsStore, requestedSelinuxMode imagecustomizerapi.SELinuxMode, + isoConfig *imagecustomizerapi.Iso, pxeConfig *imagecustomizerapi.Pxe, rawImageFile string, outputFormat imagecustomizerapi.ImageFormatType, + outputPath string, +) (err error) { + logger.Log.Infof("Creating Live OS artifacts using customized full OS image") + + liveosConfig, err := buildLiveOSConfig(outputFormat, isoConfig, pxeConfig) + if err != nil { + return fmt.Errorf("failed to build live OS configuration from input configuration:\n%w", err) + } - var extraCommandLine []string - var additionalIsoFiles imagecustomizerapi.AdditionalFileList - if isoConfig != nil { - extraCommandLine = isoConfig.KernelCommandLine.ExtraCommandLine - additionalIsoFiles = isoConfig.AdditionalFiles + err = createLiveOSFromRawHelper(buildDir, baseConfigPath, inputArtifactsStore, requestedSelinuxMode, liveosConfig, rawImageFile, outputFormat, outputPath) + if err != nil { + return fmt.Errorf("failed to create live OS artifacts:\n%w", err) } - pxeIsoImageBaseUrl := "" - if pxeConfig != nil { - pxeIsoImageBaseUrl = pxeConfig.IsoImageBaseUrl + return nil +} + +func repackageLiveOS(isoBuildDir string, baseConfigPath string, isoConfig *imagecustomizerapi.Iso, pxeConfig *imagecustomizerapi.Pxe, + inputArtifactsStore *IsoArtifactsStore, outputFormat imagecustomizerapi.ImageFormatType, outputPath string, +) error { + logger.Log.Infof("Creating Live OS artifacts using input ISO image") + + liveosConfig, err := buildLiveOSConfig(outputFormat, isoConfig, pxeConfig) + if err != nil { + return fmt.Errorf("failed to build live OS configuration from input configuration:\n%w", err) } - pxeIsoImageFileUrl := "" - if pxeConfig != nil { - pxeIsoImageFileUrl = pxeConfig.IsoImageFileUrl + err = repackageLiveOSHelper(isoBuildDir, baseConfigPath, liveosConfig, inputArtifactsStore, outputFormat, outputPath) + if err != nil { + return fmt.Errorf("failed to create live OS artifacts:\n%w", err) } + return nil +} + +func createLiveOSFromRawHelper(buildDir, baseConfigPath string, inputArtifactsStore *IsoArtifactsStore, requestedSelinuxMode imagecustomizerapi.SELinuxMode, + liveosConfig LiveOSConfig, rawImageFile string, outputFormat imagecustomizerapi.ImageFormatType, + outputPath string, +) (err error) { isoBuildDir := filepath.Join(buildDir, "liveosbuild") defer func() { cleanupErr := os.RemoveAll(isoBuildDir) @@ -71,6 +138,20 @@ func createLiveOSIsoImage(buildDir, baseConfigPath string, inputArtifactsStore * } defer rawImageConnection.Close() + // Find out if selinux is enabled + bootCustomizer, err := NewBootCustomizer(rawImageConnection.Chroot()) + if err != nil { + return fmt.Errorf("failed to attach to raw image to inspect selinux status:\n%w", err) + } + + selinuxMode, err := bootCustomizer.GetSELinuxMode(rawImageConnection.Chroot()) + if err != nil { + return fmt.Errorf("failed to get selinux mode:\n%w", err) + } + if (selinuxMode != imagecustomizerapi.SELinuxModeDisabled) && (liveosConfig.initramfsType == imagecustomizerapi.InitramfsImageTypeFullOS) { + return fmt.Errorf("selinux is not supported for full OS initramfs image") + } + // From raw image to a writeable folder writeableRootfsDir := filepath.Join(isoBuildDir, "writeable-rootfs") err = populateWriteableRootfsDir(rawImageConnection.Chroot().RootDir(), writeableRootfsDir) @@ -86,8 +167,9 @@ func createLiveOSIsoImage(buildDir, baseConfigPath string, inputArtifactsStore * } // Combine the current configuration with the saved configuration - updatedSavedConfigs, err := updateSavedConfigs(artifactsStore.files.savedConfigsFilePath, extraCommandLine, pxeIsoImageBaseUrl, - pxeIsoImageFileUrl, artifactsStore.info.dracutPackageInfo, requestedSelinuxMode, artifactsStore.info.selinuxPolicyPackageInfo) + updatedSavedConfigs, err := updateSavedConfigs(artifactsStore.files.savedConfigsFilePath, liveosConfig.kernelCommandLine, liveosConfig.bootstrapBaseUrl, + liveosConfig.bootstrapFileUrl, artifactsStore.info.kernelVersion, artifactsStore.info.dracutPackageInfo, requestedSelinuxMode, + artifactsStore.info.selinuxPolicyPackageInfo) if err != nil { return fmt.Errorf("failed to combine saved configurations with new configuration:\n%w", err) } @@ -116,13 +198,13 @@ func createLiveOSIsoImage(buildDir, baseConfigPath string, inputArtifactsStore * } // Update grub.cfg - err = updateGrubCfg(artifactsStore.files.isoGrubCfgPath, artifactsStore.files.pxeGrubCfgPath, disableSELinux, - updatedSavedConfigs, filepath.Base(outputImagePath)) + err = updateGrubCfg(outputFormat, liveosConfig.initramfsType, artifactsStore.files.isoGrubCfgPath, disableSELinux, + updatedSavedConfigs) if err != nil { return fmt.Errorf("failed to update grub.cfg:\n%w", err) } - // Generate the ISO bootimage (/boot/grub2/efiboot.img) + // Generate the ISO boot image (/boot/grub2/efiboot.img) artifactsStore.files.isoBootImagePath = filepath.Join(artifactsStore.files.artifactsDir, isoBootImagePath) err = isogenerator.BuildIsoBootImage(isoBuildDir, artifactsStore.files.bootEfiPath, artifactsStore.files.grubEfiPath, artifactsStore.files.isoBootImagePath) @@ -130,60 +212,64 @@ func createLiveOSIsoImage(buildDir, baseConfigPath string, inputArtifactsStore * return fmt.Errorf("failed to build iso boot image:\n%w", err) } - // Generate the initrd image outputInitrdPath := filepath.Join(artifactsStore.files.artifactsDir, initrdImage) - err = createInitrdImage(writeableRootfsDir, artifactsStore.info.kernelVersion, outputInitrdPath) - if err != nil { - return fmt.Errorf("failed to create initrd image:\n%w", err) - } - artifactsStore.files.initrdImagePath = outputInitrdPath - // Generate the squashfs image - outputSquashfsPath := filepath.Join(artifactsStore.files.artifactsDir, liveOSImage) - err = createSquashfsImage(writeableRootfsDir, outputSquashfsPath) - if err != nil { - return fmt.Errorf("failed to create squashfs image:\n%w", err) + switch liveosConfig.initramfsType { + case imagecustomizerapi.InitramfsImageTypeFullOS: + // Generate the initrd image + err = createFullOSInitrdImage(writeableRootfsDir, outputInitrdPath) + if err != nil { + return fmt.Errorf("failed to create initrd image:\n%w", err) + } + artifactsStore.files.initrdImagePath = outputInitrdPath + case imagecustomizerapi.InitramfsImageTypeBootstrap: + // Generate the initrd image + err = createBootstrapInitrdImage(writeableRootfsDir, artifactsStore.info.kernelVersion, outputInitrdPath) + if err != nil { + return fmt.Errorf("failed to create initrd image:\n%w", err) + } + artifactsStore.files.initrdImagePath = outputInitrdPath + + // Generate the squashfs image + outputSquashfsPath := filepath.Join(artifactsStore.files.artifactsDir, liveOSImage) + err = createSquashfsImage(writeableRootfsDir, outputSquashfsPath) + if err != nil { + return fmt.Errorf("failed to create squashfs image:\n%w", err) + } + artifactsStore.files.squashfsImagePath = outputSquashfsPath + default: + return fmt.Errorf("unsupported initramfs type (%s)", liveosConfig.initramfsType) } - artifactsStore.files.squashfsImagePath = outputSquashfsPath - // Generate the final iso image - err = createIsoImageAndPXEFolder(isoBuildDir, baseConfigPath, additionalIsoFiles, artifactsStore, outputImagePath, outputPXEArtifactsDir) - if err != nil { - return fmt.Errorf("failed to generate iso image and/or PXE artifacts folder\n%w", err) + // Generate the final output artifacts + switch outputFormat { + case imagecustomizerapi.ImageFormatTypeIso: + err := createIsoImage(isoBuildDir, baseConfigPath, artifactsStore.files, liveosConfig.additionalFiles, outputPath) + if err != nil { + return fmt.Errorf("failed to create the Iso image.\n%w", err) + } + case imagecustomizerapi.ImageFormatTypePxe: + err = createPXEArtifacts(isoBuildDir, baseConfigPath, liveosConfig.initramfsType, artifactsStore, + liveosConfig.additionalFiles, liveosConfig.bootstrapBaseUrl, liveosConfig.bootstrapFileUrl, outputPath) + if err != nil { + return fmt.Errorf("failed to generate iso image and/or PXE artifacts folder\n%w", err) + } } return nil } -func createImageFromUnchangedOS(isoBuildDir string, baseConfigPath string, isoConfig *imagecustomizerapi.Iso, - pxeConfig *imagecustomizerapi.Pxe, inputArtifactsStore *IsoArtifactsStore, outputImagePath string, outputPXEArtifactsDir string) error { - - logger.Log.Infof("Creating LiveOS iso image using unchanged OS partitions") - - var extraCommandLine []string - var additionalIsoFiles imagecustomizerapi.AdditionalFileList - if isoConfig != nil { - extraCommandLine = isoConfig.KernelCommandLine.ExtraCommandLine - additionalIsoFiles = isoConfig.AdditionalFiles - } - - pxeIsoImageBaseUrl := "" - if pxeConfig != nil { - pxeIsoImageBaseUrl = pxeConfig.IsoImageBaseUrl - } - - pxeIsoImageFileUrl := "" - if pxeConfig != nil { - pxeIsoImageFileUrl = pxeConfig.IsoImageFileUrl - } - +func repackageLiveOSHelper(isoBuildDir string, baseConfigPath string, liveosConfig LiveOSConfig, inputArtifactsStore *IsoArtifactsStore, + outputFormat imagecustomizerapi.ImageFormatType, outputPath string, +) error { // Note that in this ISO build flow, there is no os configuration, and hence // no selinux configuration. So, we will set it to default (i.e. unspecified) // and let any saved data override if present. requestedSelinuxMode := imagecustomizerapi.SELinuxModeDefault - updatedSavedConfigs, err := updateSavedConfigs(inputArtifactsStore.files.savedConfigsFilePath, extraCommandLine, pxeIsoImageBaseUrl, - pxeIsoImageFileUrl, nil /*dracut pkg info*/, requestedSelinuxMode, nil /*selinux policy pkg info*/) + updatedSavedConfigs, err := updateSavedConfigs(inputArtifactsStore.files.savedConfigsFilePath, liveosConfig.kernelCommandLine, + liveosConfig.bootstrapBaseUrl, liveosConfig.bootstrapFileUrl, inputArtifactsStore.info.kernelVersion, + nil /*dracut pkg info*/, requestedSelinuxMode, nil /*selinux policy pkg info*/) if err != nil { return fmt.Errorf("failed to combine saved configurations with new configuration:\n%w", err) } @@ -196,96 +282,26 @@ func createImageFromUnchangedOS(isoBuildDir string, baseConfigPath string, isoCo disableSELinux := false // Update grub.cfg - err = updateGrubCfg(inputArtifactsStore.files.isoGrubCfgPath, inputArtifactsStore.files.pxeGrubCfgPath, disableSELinux, updatedSavedConfigs, filepath.Base(outputImagePath)) + err = updateGrubCfg(outputFormat, liveosConfig.initramfsType, inputArtifactsStore.files.isoGrubCfgPath, + disableSELinux, updatedSavedConfigs) if err != nil { return fmt.Errorf("failed to update grub.cfg:\n%w", err) } // Generate the final iso image - err = createIsoImageAndPXEFolder(isoBuildDir, baseConfigPath, additionalIsoFiles, inputArtifactsStore, outputImagePath, outputPXEArtifactsDir) - if err != nil { - return fmt.Errorf("failed to generate iso image and/or PXE artifacts folder\n%w", err) - } - - return nil -} - -func createIsoImageAndPXEFolder(buildDir string, baseConfigPath string, additionalIsoFiles imagecustomizerapi.AdditionalFileList, artifactsStore *IsoArtifactsStore, outputImagePath string, - outputPXEArtifactsDir string) error { - - err := createIsoImage(buildDir, artifactsStore.files, baseConfigPath, additionalIsoFiles, outputImagePath) - if err != nil { - return fmt.Errorf("failed to create the Iso image.\n%w", err) - } - - if outputPXEArtifactsDir != "" { - err = verifyDracutPXESupport(artifactsStore.info.dracutPackageInfo) + switch outputFormat { + case imagecustomizerapi.ImageFormatTypeIso: + err := createIsoImage(isoBuildDir, baseConfigPath, inputArtifactsStore.files, liveosConfig.additionalFiles, outputPath) if err != nil { - return fmt.Errorf("failed to verify Dracut's PXE support.\n%w", err) + return fmt.Errorf("failed to create the Iso image.\n%w", err) } - err = populatePXEArtifactsDir(outputImagePath, buildDir, outputPXEArtifactsDir) + case imagecustomizerapi.ImageFormatTypePxe: + err = createPXEArtifacts(isoBuildDir, baseConfigPath, liveosConfig.initramfsType, inputArtifactsStore, + liveosConfig.additionalFiles, liveosConfig.bootstrapBaseUrl, liveosConfig.bootstrapFileUrl, outputPath) if err != nil { - return fmt.Errorf("failed to populate the PXE artifacts folder.\n%w", err) + return fmt.Errorf("failed to generate iso image and/or PXE artifacts folder\n%w", err) } } return nil } - -func populatePXEArtifactsDir(isoImagePath string, buildDir string, outputPXEArtifactsDir string) error { - - logger.Log.Infof("Copying PXE artifacts to (%s)", outputPXEArtifactsDir) - - // Extract all files from the iso image file. - err := extractIsoImageContents(buildDir, isoImagePath, outputPXEArtifactsDir) - if err != nil { - return err - } - - // Replace the iso grub.cfg with the PXE grub.cfg - isoGrubCfgPath := filepath.Join(outputPXEArtifactsDir, grubCfgDir, isoGrubCfg) - pxeGrubCfgPath := filepath.Join(outputPXEArtifactsDir, grubCfgDir, pxeGrubCfg) - err = file.Copy(pxeGrubCfgPath, isoGrubCfgPath) - if err != nil { - return fmt.Errorf("failed to copy (%s) to (%s) while populating the PXE artifacts directory:\n%w", pxeGrubCfgPath, isoGrubCfgPath, err) - } - - err = os.RemoveAll(pxeGrubCfgPath) - if err != nil { - return fmt.Errorf("failed to remove file (%s):\n%w", pxeGrubCfgPath, err) - } - - _, bootFilesConfig, err := getBootArchConfig() - if err != nil { - return err - } - // Move bootloader files from under '/efi/boot' to '/' - bootloaderSrcDir := filepath.Join(outputPXEArtifactsDir, isoBootloadersDir) - bootloaderFiles := []string{bootFilesConfig.bootBinary, bootFilesConfig.grubBinary} - - for _, bootloaderFile := range bootloaderFiles { - sourcePath := filepath.Join(bootloaderSrcDir, bootloaderFile) - targetPath := filepath.Join(outputPXEArtifactsDir, bootloaderFile) - err = file.Move(sourcePath, targetPath) - if err != nil { - return fmt.Errorf("failed to move boot loader file from (%s) to (%s) while generated the PXE artifacts folder:\n%w", sourcePath, targetPath, err) - } - } - - // Remove the empty 'pxe-folder>/efi' folder. - isoEFIDir := filepath.Join(outputPXEArtifactsDir, "efi") - err = os.RemoveAll(isoEFIDir) - if err != nil { - return fmt.Errorf("failed to remove folder (%s):\n%w", isoEFIDir, err) - } - - // The iso image file itself must be placed in the PXE folder because - // dracut livenet module will download it. - artifactsIsoImagePath := filepath.Join(outputPXEArtifactsDir, filepath.Base(isoImagePath)) - err = file.Copy(isoImagePath, artifactsIsoImagePath) - if err != nil { - return fmt.Errorf("failed to copy (%s) while populating the PXE artifacts directory:\n%w", isoImagePath, err) - } - - return nil -} diff --git a/toolkit/tools/pkg/imagecustomizerlib/liveosisobuilder_test.go b/toolkit/tools/pkg/imagecustomizerlib/liveosisobuilder_test.go index 8736c6b8cd..8d51f386a7 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/liveosisobuilder_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/liveosisobuilder_test.go @@ -13,273 +13,472 @@ import ( "github.com/microsoft/azurelinux/toolkit/tools/imagecustomizerapi" "github.com/microsoft/azurelinux/toolkit/tools/internal/file" + "github.com/microsoft/azurelinux/toolkit/tools/internal/initrdutils" + "github.com/microsoft/azurelinux/toolkit/tools/internal/logger" "github.com/microsoft/azurelinux/toolkit/tools/internal/safeloopback" "github.com/microsoft/azurelinux/toolkit/tools/internal/safemount" + "github.com/microsoft/azurelinux/toolkit/tools/internal/tarutils" + "github.com/stretchr/testify/assert" "golang.org/x/sys/unix" ) -// Tests: -// - vhdx to ISO, with OS changes, and PXE image base URL. -// - ISO to ISO, with no OS changes. -// - Kernel command-line arg append. -// - .iso.additionalFiles -func TestCustomizeImageLiveCd1(t *testing.T) { - baseImage := checkSkipForCustomizeImage(t, baseImageTypeCoreEfi, baseImageVersionDefault) +func createConfig(fileName, kernelParameter string, initramfsType imagecustomizerapi.InitramfsImageType, bootstrapFileUrl string, + enableOsConfig, bootstrapPrereqs bool, selinuxMode imagecustomizerapi.SELinuxMode) *imagecustomizerapi.Config { + + bootstrapRequiredPkgs := []string{} + if bootstrapPrereqs { + bootstrapRequiredPkgs = []string{ + "squashfs-tools", + "tar", + "device-mapper", + "curl", + } + } - testTempDir := filepath.Join(tmpDir, "TestCustomizeImageLiveCd1") - buildDir := filepath.Join(testTempDir, "build") - outImageFileName := "image.iso" - outImageFilePath := filepath.Join(testTempDir, outImageFileName) - pxeArtifactsPathVhdxToIso := "" - pxeArtifactsPathIsoToIso := "" - if baseImageVersionDefault != baseImageVersionAzl2 { - pxeArtifactsPathVhdxToIso = filepath.Join(testTempDir, "pxe-artifacts-vhdx-to-iso") - pxeArtifactsPathIsoToIso = filepath.Join(testTempDir, "pxe-artifacts-iso-to-iso") + if selinuxMode != imagecustomizerapi.SELinuxModeDisabled { + bootstrapRequiredPkgs = append(bootstrapRequiredPkgs, "selinux-policy") } - pxeKernelIpArg := "linux.* ip=dhcp " - pxeImageFileUrlV1, err := url.JoinPath("http://my-pxe-server-1/", outImageFileName) - assert.NoError(t, err) - pxeKernelRootArgV1 := "linux.* root=live:" + pxeImageFileUrlV1 - pxeKernelRootArgV1 = strings.ReplaceAll(pxeKernelRootArgV1, "/", "\\/") - pxeKernelRootArgV1 = strings.ReplaceAll(pxeKernelRootArgV1, ":", "\\:") - configFile := filepath.Join(testDir, "iso-files-and-args-config.yaml") + perms0o644 := imagecustomizerapi.FilePermissions(0o644) - // Customize vhdx to ISO, with OS changes. - err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "iso", - pxeArtifactsPathVhdxToIso, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) - assert.NoError(t, err) + config := imagecustomizerapi.Config{ + Iso: &imagecustomizerapi.Iso{ + AdditionalFiles: imagecustomizerapi.AdditionalFileList{ + { + Source: filepath.Join("files/", fileName), + Destination: filepath.Join("/", fileName), + Permissions: &perms0o644, + }, + }, + KernelCommandLine: imagecustomizerapi.KernelCommandLine{ + ExtraCommandLine: []string{kernelParameter}, + }, + InitramfsType: initramfsType, + }, + Pxe: &imagecustomizerapi.Pxe{ + AdditionalFiles: imagecustomizerapi.AdditionalFileList{ + { + Source: filepath.Join("files/", fileName), + Destination: filepath.Join("/", fileName), + Permissions: &perms0o644, + }, + }, + KernelCommandLine: imagecustomizerapi.KernelCommandLine{ + ExtraCommandLine: []string{kernelParameter}, + }, + InitramfsType: initramfsType, + BootstrapFileUrl: bootstrapFileUrl, + }, + } - // Attach ISO. - isoImageLoopDevice, err := safeloopback.NewLoopback(outImageFilePath) - if !assert.NoError(t, err) { - return + if enableOsConfig { + config.OS = &imagecustomizerapi.OS{ + AdditionalFiles: imagecustomizerapi.AdditionalFileList{ + { + Source: filepath.Join("files/", fileName), + Destination: filepath.Join("/", fileName), + // Need to ensure the packaged full OS supports setuid, setgid, and sticky bits. + Permissions: &perms0o644, + }, + }, + SELinux: imagecustomizerapi.SELinux{ + Mode: selinuxMode, + }, + Packages: imagecustomizerapi.Packages{ + Install: bootstrapRequiredPkgs, + }, + } } - defer isoImageLoopDevice.Close() - isoMountDir := filepath.Join(testTempDir, "iso-mount") - isoImageMount, err := safemount.NewMount(isoImageLoopDevice.DevicePath(), isoMountDir, - "iso9660" /*fstype*/, unix.MS_RDONLY /*flags*/, "" /*data*/, true /*makeAndDelete*/) - if !assert.NoError(t, err) { - return + return &config +} + +func VerifyBootstrapPresence(t *testing.T, initramfsType imagecustomizerapi.InitramfsImageType, bootstrapImagePath string) { + + bootstrapImageExists, err := file.PathExists(bootstrapImagePath) + assert.NoErrorf(t, err, "check if (%s) bootstrapImagePath exists", bootstrapImagePath) + + switch initramfsType { + case imagecustomizerapi.InitramfsImageTypeBootstrap: + assert.Equal(t, bootstrapImageExists, true) + case imagecustomizerapi.InitramfsImageTypeFullOS: + assert.Equal(t, bootstrapImageExists, false) } - defer isoImageMount.Close() +} + +func ValidateLiveOSContent(t *testing.T, config *imagecustomizerapi.Config, testTempDir, outputFormat string, artifactsPath, bootstrappedImage string) { // Check for the copied a.txt file. - aOrigPath := filepath.Join(testDir, "files/a.txt") - aIsoPath := filepath.Join(isoMountDir, "a.txt") - verifyFileContentsSame(t, aOrigPath, aIsoPath) + var additionalFiles imagecustomizerapi.AdditionalFileList + var extraCommandLineParameters []string + var initramfsType imagecustomizerapi.InitramfsImageType + var pxeUrlBase string + + if outputFormat == "iso" { + additionalFiles = config.Iso.AdditionalFiles + extraCommandLineParameters = config.Iso.KernelCommandLine.ExtraCommandLine + initramfsType = config.Iso.InitramfsType + } else { + additionalFiles = config.Pxe.AdditionalFiles + extraCommandLineParameters = config.Pxe.KernelCommandLine.ExtraCommandLine + initramfsType = config.Pxe.InitramfsType + pxeUrlBase = config.Pxe.BootstrapFileUrl + } + + for _, additionalFile := range additionalFiles { + origFilePath := filepath.Join(testDir, additionalFile.Source) + fullOSFilePath := filepath.Join(artifactsPath, additionalFile.Destination) + verifyFileContentsSame(t, origFilePath, fullOSFilePath) + verifyFilePermissions(t, os.FileMode(*additionalFile.Permissions), fullOSFilePath) + } // Ensure grub.cfg file has the extra kernel command-line args. - grubCfgFilePath := filepath.Join(isoMountDir, "/boot/grub2/grub.cfg") + grubCfgFilePath := filepath.Join(artifactsPath, grubCfgDir, isoGrubCfg) grubCfgContents, err := file.Read(grubCfgFilePath) assert.NoError(t, err, "read grub.cfg file") - assert.Regexp(t, "linux.* rd.info ", grubCfgContents) + for _, extraCommandLineParameter := range extraCommandLineParameters { + assert.Regexp(t, "linux.* "+extraCommandLineParameter+" ", grubCfgContents) + } // Check the saved-configs.yaml file. - savedConfigsFilePath := filepath.Join(isoMountDir, savedConfigsDir, savedConfigsFileName) + savedConfigsFilePath := filepath.Join(artifactsPath, savedConfigsDir, savedConfigsFileName) savedConfigs := &SavedConfigs{} err = imagecustomizerapi.UnmarshalAndValidateYamlFile(savedConfigsFilePath, savedConfigs) assert.NoErrorf(t, err, "read (%s) file", savedConfigsFilePath) - expectedKernelArgs := []string{"rd.info"} - assert.Equal(t, expectedKernelArgs, savedConfigs.Iso.KernelCommandLine.ExtraCommandLine) + for _, extraCommandLineParameter := range extraCommandLineParameters { + assert.Contains(t, savedConfigs.Iso.KernelCommandLine.ExtraCommandLine, extraCommandLineParameter) + } - VerifyPXEArtifacts(t, savedConfigs.OS.DracutPackageInfo, isoMountDir, pxeKernelIpArg, pxeKernelRootArgV1, - pxeArtifactsPathVhdxToIso) + bootstrapImagePath := "" + if outputFormat == "iso" { + // The bootstrap file is a squashfs image file + bootstrapImagePath = filepath.Join(artifactsPath, liveOSDir, liveOSImage) + } else { + // The bootstrap file is an iso that contains the squashfs file + bootstrapImagePath = filepath.Join(artifactsPath, defaultIsoImageName) + } - err = isoImageMount.CleanClose() - if !assert.NoError(t, err) { - return + VerifyBootstrapPresence(t, initramfsType, bootstrapImagePath) + VerifyFullOSContents(t, testTempDir, artifactsPath, outputFormat, config.OS, bootstrapImagePath, initramfsType) + + if outputFormat == "pxe" { + if initramfsType == imagecustomizerapi.InitramfsImageTypeBootstrap { + VerifyBootstrapPXEArtifacts(t, savedConfigs.OS.DracutPackageInfo, filepath.Base(bootstrappedImage), artifactsPath, pxeUrlBase) + } } +} - err = isoImageLoopDevice.CleanClose() - if !assert.NoError(t, err) { +func VerifyFullOSContents(t *testing.T, testTempDir, artifactsPath, outputFormat string, osConfig *imagecustomizerapi.OS, bootstrapImagePath string, initramfsType imagecustomizerapi.InitramfsImageType) { + if osConfig == nil { return } + fullOsDir := filepath.Join(testTempDir, "full-os") + + switch initramfsType { + case imagecustomizerapi.InitramfsImageTypeBootstrap: + fullOSImagePath := "" + if outputFormat == "iso" { + // The full OS image is the bootstrap image + fullOSImagePath = bootstrapImagePath + } else { + // The bootstrap file is an iso that contains the squashfs file + isoImageLoopDevice, err := safeloopback.NewLoopback(bootstrapImagePath) + if !assert.NoError(t, err) { + return + } + defer isoImageLoopDevice.Close() + + isoMountDir := filepath.Join(testTempDir, "bootstrap-iso-mount") + isoImageMount, err := safemount.NewMount(isoImageLoopDevice.DevicePath(), isoMountDir, + "iso9660" /*fstype*/, unix.MS_RDONLY /*flags*/, "" /*data*/, true /*makeAndDelete*/) + if !assert.NoError(t, err) { + return + } + defer isoImageMount.Close() + + fullOSImagePath = filepath.Join(isoMountDir, liveOSDir, liveOSImage) + } + + // Attach squashfs file. + squashfsLoopDevice, err := safeloopback.NewLoopback(fullOSImagePath) + if !assert.NoError(t, err) { + return + } + defer squashfsLoopDevice.Close() + + squashfsMount, err := safemount.NewMount(squashfsLoopDevice.DevicePath(), fullOsDir, + "squashfs" /*fstype*/, unix.MS_RDONLY /*flags*/, "" /*data*/, true /*makeAndDelete*/) + if !assert.NoError(t, err) { + return + } + defer squashfsMount.Close() + case imagecustomizerapi.InitramfsImageTypeFullOS: + fullOSImagePath := filepath.Join(artifactsPath, "boot/initrd.img") + // Expand initrd to a folder + err := initrdutils.CreateFolderFromInitrdImage(fullOSImagePath, fullOsDir) + if !assert.NoError(t, err) { + return + } + defer os.RemoveAll(fullOsDir) + } - // Customize ISO to ISO, with no OS changes. - pxeImageFileUrlV2, err := url.JoinPath("http://my-pxe-server-2/", outImageFileName) - assert.NoError(t, err) - - pxeKernelRootArgV2 := "linux.* root=live:" + pxeImageFileUrlV2 - pxeKernelRootArgV2 = strings.ReplaceAll(pxeKernelRootArgV2, "/", "\\/") - pxeKernelRootArgV2 = strings.ReplaceAll(pxeKernelRootArgV2, ":", "\\:") - - b2FilePerms := imagecustomizerapi.FilePermissions(0o600) - config := imagecustomizerapi.Config{ - Pxe: &imagecustomizerapi.Pxe{ - IsoImageFileUrl: pxeImageFileUrlV2, - }, - Iso: &imagecustomizerapi.Iso{ - KernelCommandLine: imagecustomizerapi.KernelCommandLine{ - ExtraCommandLine: []string{"rd.debug"}, - }, - AdditionalFiles: imagecustomizerapi.AdditionalFileList{ - { - Source: "files/b.txt", - Destination: "/b1.txt", - }, - { - Source: "files/b.txt", - Destination: "/b2.txt", - Permissions: &b2FilePerms, - }, - }, - }, + // Check that each file is in the root file system. + for _, additionalFile := range osConfig.AdditionalFiles { + origFilePath := filepath.Join(testDir, additionalFile.Source) + fullOSFilePath := filepath.Join(fullOsDir, additionalFile.Destination) + verifyFileContentsSame(t, origFilePath, fullOSFilePath) + verifyFilePermissions(t, os.FileMode(*additionalFile.Permissions), fullOSFilePath) } - err = CustomizeImage(buildDir, testDir, &config, outImageFilePath, nil, outImageFilePath, "iso", - pxeArtifactsPathIsoToIso, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) - assert.NoError(t, err) +} - // Attach ISO. - isoImageLoopDevice, err = safeloopback.NewLoopback(outImageFilePath) +func ValidateIsoContent(t *testing.T, config *imagecustomizerapi.Config, testTempDir string, initramfsType imagecustomizerapi.InitramfsImageType, outImageFilePath string) { + isoImageLoopDevice, err := safeloopback.NewLoopback(outImageFilePath) if !assert.NoError(t, err) { return } defer isoImageLoopDevice.Close() - isoImageMount, err = safemount.NewMount(isoImageLoopDevice.DevicePath(), isoMountDir, + isoMountDir := filepath.Join(testTempDir, "iso-mount") + isoImageMount, err := safemount.NewMount(isoImageLoopDevice.DevicePath(), isoMountDir, "iso9660" /*fstype*/, unix.MS_RDONLY /*flags*/, "" /*data*/, true /*makeAndDelete*/) if !assert.NoError(t, err) { return } defer isoImageMount.Close() - // Check that the a.txt stayed around. - verifyFileContentsSame(t, aOrigPath, aIsoPath) - - // Check for copied b.txt file. - bOrigPath := filepath.Join(testDir, "files/b.txt") - b1IsoPath := filepath.Join(isoMountDir, "b1.txt") - b2IsoPath := filepath.Join(isoMountDir, "b2.txt") - verifyFileContentsSame(t, bOrigPath, b1IsoPath) - verifyFileContentsSame(t, bOrigPath, b2IsoPath) - verifyFilePermissions(t, os.FileMode(b2FilePerms), b2IsoPath) + ValidateLiveOSContent(t, config, testTempDir, "iso" /*outputFormat*/, isoMountDir, "" /*bootstrappedImage*/) +} - // Ensure grub.cfg file has the extra kernel command-line args from both runs. - grubCfgContents, err = file.Read(grubCfgFilePath) - assert.NoError(t, err, "read grub.cfg file") - assert.Regexp(t, "linux.* rd.info ", grubCfgContents) - assert.Regexp(t, "linux.* rd.debug ", grubCfgContents) +func ValidatePxeContent(t *testing.T, config *imagecustomizerapi.Config, testTempDir, outImageFilePath string) { + pxeArtifactsPath := "" + if strings.HasSuffix(outImageFilePath, ".tar.gz") { + pxeArtifactsPath = filepath.Join(testTempDir, "pxe-artifacts") + err := tarutils.ExpandTarGzArchive(outImageFilePath, pxeArtifactsPath) + if !assert.NoError(t, err) { + return + } + } else { + pxeArtifactsPath = outImageFilePath + } - // Check the iso-kernel-args.txt file. - savedConfigs = &SavedConfigs{} - err = imagecustomizerapi.UnmarshalAndValidateYamlFile(savedConfigsFilePath, savedConfigs) - assert.NoErrorf(t, err, "read (%s) file", savedConfigsFilePath) - assert.Equal(t, []string{"rd.info", "rd.debug"}, savedConfigs.Iso.KernelCommandLine.ExtraCommandLine) + bootstrappedImage := "" + if config.Pxe != nil && config.Pxe.InitramfsType == imagecustomizerapi.InitramfsImageTypeBootstrap { + bootstrappedImage = filepath.Join(pxeArtifactsPath, defaultIsoImageName) + } - VerifyPXEArtifacts(t, savedConfigs.OS.DracutPackageInfo, isoMountDir, pxeKernelIpArg, pxeKernelRootArgV2, - pxeArtifactsPathIsoToIso) + ValidateLiveOSContent(t, config, testTempDir, "pxe" /*outputFormat*/, pxeArtifactsPath, bootstrappedImage) } -func VerifyPXEArtifacts(t *testing.T, packageInfo *PackageVersionInformation, isoMountDir string, pxeKernelIpArg string, - pxeKernelRootArgV2 string, pxeArtifactsPathIsoToIso string) { +func VerifyBootstrapPXEArtifacts(t *testing.T, packageInfo *PackageVersionInformation, outImageFileName, isoMountDir, pxeBaseUrl string) { + var err error + pxeKernelIpArg := "linux.* ip=dhcp " + + pxeImageFileUrl := "" + if strings.HasSuffix(pxeBaseUrl, ".iso") { + pxeImageFileUrl = pxeBaseUrl + } else { + pxeImageFileUrl, err = url.JoinPath(pxeBaseUrl, outImageFileName) + assert.NoError(t, err) + } + + pxeKernelRootArg := "linux.* root=live:" + pxeImageFileUrl + pxeKernelRootArg = strings.ReplaceAll(pxeKernelRootArg, "/", "\\/") + pxeKernelRootArg = strings.ReplaceAll(pxeKernelRootArg, ":", "\\:") // Check if PXE support is present in the Dracut package version in use. - err := verifyDracutPXESupport(packageInfo) + err = verifyDracutPXESupport(packageInfo) if err != nil { - // If there is no PXE support, return + // If there is not PXE support, return + logger.Log.Infof("PXE is not supported for this Dracut version - skipping validation") return } // Ensure grub-pxe.cfg file exists and has the pxe-specific command-line args. - pxeGrubCfgFilePath := filepath.Join(isoMountDir, "/boot/grub2/grub-pxe.cfg") + pxeGrubCfgFilePath := filepath.Join(isoMountDir, "/boot/grub2/grub.cfg") pxeGrubCfgContents, err := file.Read(pxeGrubCfgFilePath) - assert.NoError(t, err, "read grub-pxe.cfg file") + assert.NoError(t, err, "read grub.cfg file") assert.Regexp(t, pxeKernelIpArg, pxeGrubCfgContents) - assert.Regexp(t, pxeKernelRootArgV2, pxeGrubCfgContents) + assert.Regexp(t, pxeKernelRootArg, pxeGrubCfgContents) +} + +// Tests: +// - raw -> iso {bootstrap} -> iso {bootstrap} -> iso {full-os} +// +// - vhdx {raw} to ISO {bootstrap} , with selinux enforcing + bootstrap prereqs +// - ISO {bootstrap} to ISO {bootstrap} , with no OS changes +// - ISO {bootstrap} to ISO {full-os} , with selinux disabled +func TestCustomizeImageLiveOSInitramfs1(t *testing.T) { + baseImage := checkSkipForCustomizeImage(t, baseImageTypeCoreEfi, baseImageVersionDefault) + + testTempDir := filepath.Join(tmpDir, "TestCustomizeImageLiveOSInitramfs1") + buildDir := filepath.Join(testTempDir, "build") + outImageFilePath := filepath.Join(testTempDir, defaultIsoImageName) + + configA := createConfig("a.txt", "rd.info", imagecustomizerapi.InitramfsImageTypeBootstrap, "", /*pxe url*/ + true /*enable os config*/, true /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeEnforcing) + + // vhdx {raw} to ISO {bootstrap}, selinux enforcing + bootstrap prereqs + err := CustomizeImage(buildDir, testDir, configA, baseImage, nil, outImageFilePath, "iso", + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + if !assert.NoError(t, err) { + return + } + + ValidateIsoContent(t, configA, testTempDir, imagecustomizerapi.InitramfsImageTypeBootstrap, outImageFilePath) + + // ISO {bootstrap} to ISO {bootstrap}, with no OS changes + configB := createConfig("b.txt", "rd.debug", imagecustomizerapi.InitramfsImageTypeBootstrap, "", /*pxe url*/ + false /*enable os config*/, false /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeDefault) + + err = CustomizeImage(buildDir, testDir, configB, outImageFilePath, nil, outImageFilePath, "iso", + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + if !assert.NoError(t, err) { + return + } + + ValidateIsoContent(t, configB, testTempDir, imagecustomizerapi.InitramfsImageTypeBootstrap, outImageFilePath) + + // - ISO {bootstrap} to ISO {full-os}, with selinux disabled + configC := createConfig("c.txt", "rd.shell", imagecustomizerapi.InitramfsImageTypeFullOS, "", /*pxe url*/ + true /*enable os config*/, false /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeDisabled) + + err = CustomizeImage(buildDir, testDir, configC, outImageFilePath, nil, outImageFilePath, "iso", + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + if !assert.NoError(t, err) { + return + } - exportedPxeGrubCfgFilePath := filepath.Join(pxeArtifactsPathIsoToIso, "boot/grub2/grub.cfg") - exportedPxeGrubCfgContents, err := file.Read(exportedPxeGrubCfgFilePath) - assert.NoError(t, err, "read pxe grub.cfg file") - assert.Equal(t, pxeGrubCfgContents, exportedPxeGrubCfgContents) + ValidateIsoContent(t, configC, testTempDir, imagecustomizerapi.InitramfsImageTypeFullOS, outImageFilePath) } // Tests: -// - vhdx to ISO, with no OS changes. -// - ISO to ISO, with OS changes. -func TestCustomizeImageLiveCd2(t *testing.T) { +// - raw -> iso {full-os} -> iso {full-os} -> iso {bootstrap} +// +// - vhdx {raw} to ISO {full-os} , with selinux disabled +// - ISO {full-os} to ISO {full-os} , with selinux disabled +// - ISO {full-os} to ISO {bootstrap} , with selinux enforcing + bootstrap prereqs +func TestCustomizeImageLiveOSInitramfs2(t *testing.T) { baseImage := checkSkipForCustomizeImage(t, baseImageTypeCoreEfi, baseImageVersionDefault) - testTempDir := filepath.Join(tmpDir, "TestCustomizeImageLiveCd2") + testTempDir := filepath.Join(tmpDir, "TestCustomizeImageLiveOSInitramfs2") buildDir := filepath.Join(testTempDir, "build") - outImageFilePath := filepath.Join(testTempDir, "image.raw") - outIsoFilePath := filepath.Join(testTempDir, "image.iso") + outImageFilePath := filepath.Join(testTempDir, defaultIsoImageName) - // Customize vhdx with ISO prereqs. - configFile := filepath.Join(testDir, "iso-os-prereqs-config.yaml") - err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) - assert.NoError(t, err) + // vhdx {raw} to ISO {full-os}, with selinux disabled + configA := createConfig("a.txt", "rd.info", imagecustomizerapi.InitramfsImageTypeFullOS, "", /*pxe url*/ + true /*enable os config*/, false /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeDisabled) - // Customize image to ISO, with no OS changes. - config := imagecustomizerapi.Config{ - Iso: &imagecustomizerapi.Iso{}, + err := CustomizeImage(buildDir, testDir, configA, baseImage, nil, outImageFilePath, "iso", + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + if !assert.NoError(t, err) { + return } - err = CustomizeImage(buildDir, testDir, &config, outImageFilePath, nil, outIsoFilePath, "iso", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) - assert.NoError(t, err) - // Customize ISO to ISO, with OS changes. - configFile = filepath.Join(testDir, "addfiles-config.yaml") - err = CustomizeImageWithConfigFile(buildDir, configFile, outIsoFilePath, nil, outIsoFilePath, "iso", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) - assert.NoError(t, err) + ValidateIsoContent(t, configA, testTempDir, imagecustomizerapi.InitramfsImageTypeFullOS, outImageFilePath) - // Attach ISO. - isoImageLoopDevice, err := safeloopback.NewLoopback(outIsoFilePath) + // ISO {full-os} to ISO {full-os}, with selinux disabled + configB := createConfig("b.txt", "rd.shell", imagecustomizerapi.InitramfsImageTypeFullOS, "", /*pxe url*/ + true /*enable os config*/, true /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeDisabled) + + err = CustomizeImage(buildDir, testDir, configB, outImageFilePath, nil, outImageFilePath, "iso", + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } - defer isoImageLoopDevice.Close() - isoMountDir := filepath.Join(testTempDir, "iso-mount") - isoImageMount, err := safemount.NewMount(isoImageLoopDevice.DevicePath(), isoMountDir, - "iso9660" /*fstype*/, unix.MS_RDONLY /*flags*/, "" /*data*/, true /*makeAndDelete*/) + ValidateIsoContent(t, configB, testTempDir, imagecustomizerapi.InitramfsImageTypeFullOS, outImageFilePath) + + // - ISO {full-os} to ISO {bootstrap}, with selinux enforcing + configC := createConfig("c.txt", "rd.shell", imagecustomizerapi.InitramfsImageTypeBootstrap, "", /*pxe url*/ + true /*enable os config*/, true /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeEnforcing) + + err = CustomizeImage(buildDir, testDir, configC, outImageFilePath, nil, outImageFilePath, "iso", + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } - defer isoImageMount.Close() - // Attach squashfs file. - squashfsPath := filepath.Join(isoMountDir, liveOSDir, liveOSImage) - squashfsLoopDevice, err := safeloopback.NewLoopback(squashfsPath) + ValidateIsoContent(t, configC, testTempDir, imagecustomizerapi.InitramfsImageTypeBootstrap, outImageFilePath) +} + +// Tests: +// - vhdx {raw} to ISO {full-os}, with selinux enabled -> error +func TestCustomizeImageLiveOSInitramfs3(t *testing.T) { + baseImage := checkSkipForCustomizeImage(t, baseImageTypeCoreEfi, baseImageVersionDefault) + + testTempDir := filepath.Join(tmpDir, "TestCustomizeImageLiveOSInitramfs3") + buildDir := filepath.Join(testTempDir, "build") + outImageFilePath := filepath.Join(testTempDir, defaultIsoImageName) + + // vhdx {raw} to ISO {full-os}, with selinux disabled + configA := createConfig("a.txt", "rd.info", imagecustomizerapi.InitramfsImageTypeFullOS, "", /*pxe url*/ + true /*enable os config*/, false /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeEnforcing) + + err := CustomizeImage(buildDir, testDir, configA, baseImage, nil, outImageFilePath, "iso", + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + assert.ErrorContains(t, err, "selinux is not supported for full OS initramfs image") +} + +// Tests: +// - vhdx {raw} to PXE {bootstrap}, with selinux enforcing +func TestCustomizeImageLiveOSPxe1(t *testing.T) { + baseImage := checkSkipForCustomizeImage(t, baseImageTypeCoreEfi, baseImageVersionDefault) + + testTempDir := filepath.Join(tmpDir, "TestCustomizeImageLiveOSPxe1") + buildDir := filepath.Join(testTempDir, "build") + outImageFilePath := filepath.Join(testTempDir, "pxe-artifacts.tar.gz") + pxeBootstrapUrl := "http://my-pxe-server-1/" + defaultIsoImageName + + config := createConfig("a.txt", "rd.info", imagecustomizerapi.InitramfsImageTypeBootstrap, pxeBootstrapUrl, + true /*enable os config*/, true /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeEnforcing) + + err := CustomizeImage(buildDir, testDir, config, baseImage, nil, outImageFilePath, "pxe", + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } - defer squashfsLoopDevice.Close() - squashfsMountDir := filepath.Join(testTempDir, "iso-squashfs") - squashfsMount, err := safemount.NewMount(squashfsLoopDevice.DevicePath(), squashfsMountDir, - "squashfs" /*fstype*/, unix.MS_RDONLY /*flags*/, "" /*data*/, true /*makeAndDelete*/) + ValidatePxeContent(t, config, testTempDir, outImageFilePath) +} + +// Tests: +// - vhdx {raw} to PXE {full-os}, with selinux disabled +func TestCustomizeImageLiveOSPxe2(t *testing.T) { + baseImage := checkSkipForCustomizeImage(t, baseImageTypeCoreEfi, baseImageVersionDefault) + + testTempDir := filepath.Join(tmpDir, "TestCustomizeImageLiveOSPxe2") + buildDir := filepath.Join(testTempDir, "build") + outImageFilePath := filepath.Join(testTempDir, "pxe-artifacts.tar.gz") + + config := createConfig("a.txt", "rd.info", imagecustomizerapi.InitramfsImageTypeFullOS, "", /*pxe url*/ + true /*enable os config*/, false /*bootstrap prereqs*/, imagecustomizerapi.SELinuxModeDisabled) + + err := CustomizeImage(buildDir, testDir, config, baseImage, nil, outImageFilePath, "pxe", + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } - defer squashfsMount.Close() - // Check that a.txt is in the squashfs file. - aOrigPath := filepath.Join(testDir, "files/a.txt") - aIsoPath := filepath.Join(squashfsMountDir, "/mnt/a/a.txt") - verifyFileContentsSame(t, aOrigPath, aIsoPath) + ValidatePxeContent(t, config, testTempDir, outImageFilePath) } -func TestCustomizeImageLiveCdIsoNoShimEfi(t *testing.T) { +func TestCustomizeImageLiveOSIsoNoShimEfi(t *testing.T) { for _, version := range supportedAzureLinuxVersions { t.Run(string(version), func(t *testing.T) { - testCustomizeImageLiveCdIsoNoShimEfi(t, "TestCustomizeImageLiveCdIsoNoShimEfi"+string(version), + testCustomizeImageLiveOSIsoNoShimEfi(t, "TestCustomizeImageLiveCdIsoNoShimEfi"+string(version), version) }) } } -func testCustomizeImageLiveCdIsoNoShimEfi(t *testing.T, testName string, version baseImageVersion) { +func testCustomizeImageLiveOSIsoNoShimEfi(t *testing.T, testName string, version baseImageVersion) { baseImage := checkSkipForCustomizeImage(t, baseImageTypeCoreEfi, version) buildDir := filepath.Join(tmpDir, testName) - outImageFilePath := filepath.Join(buildDir, "image.iso") + outImageFilePath := filepath.Join(buildDir, defaultIsoImageName) shimPackage := "shim" // For arm64 and baseImageVersionAzl2, the shim package is shim-unsigned. @@ -299,16 +498,16 @@ func testCustomizeImageLiveCdIsoNoShimEfi(t *testing.T, testName string, version // Customize image. err := CustomizeImage(buildDir, testDir, config, baseImage, nil, outImageFilePath, "iso", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.Error(t, err) assert.ErrorContains(t, err, "failed to find the boot efi file") } -func TestCustomizeImageLiveCdIsoNoGrubEfi(t *testing.T) { +func TestCustomizeImageLiveOSIsoNoGrubEfi(t *testing.T) { baseImage := checkSkipForCustomizeImage(t, baseImageTypeCoreEfi, baseImageVersionDefault) - buildDir := filepath.Join(tmpDir, "TestCustomizeImageLiveCdIso") - outImageFilePath := filepath.Join(buildDir, "image.iso") + buildDir := filepath.Join(tmpDir, "TestCustomizeImageLiveOSIsoNoGrubEfi") + outImageFilePath := filepath.Join(buildDir, defaultIsoImageName) config := &imagecustomizerapi.Config{ OS: &imagecustomizerapi.OS{ @@ -322,7 +521,7 @@ func TestCustomizeImageLiveCdIsoNoGrubEfi(t *testing.T) { // Customize image. err := CustomizeImage(buildDir, testDir, config, baseImage, nil, outImageFilePath, "iso", - "" /*outputPXEArtifactsDir*/, true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + true /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) assert.Error(t, err) assert.ErrorContains(t, err, "failed to find the grub efi file") } diff --git a/toolkit/tools/pkg/imagecustomizerlib/liveosisogrub.go b/toolkit/tools/pkg/imagecustomizerlib/liveosisogrub.go index 9ba4d2aad8..8aa9333b73 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/liveosisogrub.go +++ b/toolkit/tools/pkg/imagecustomizerlib/liveosisogrub.go @@ -5,7 +5,6 @@ package imagecustomizerlib import ( "fmt" - "net/url" "github.com/microsoft/azurelinux/toolkit/tools/imagecustomizerapi" "github.com/microsoft/azurelinux/toolkit/tools/internal/file" @@ -29,18 +28,12 @@ const ( liveOSImagePath = "/" + liveOSDir + "/" + liveOSImage ) -func updateGrubCfg(isoGrubCfgFileName string, pxeGrubCfgFileName string, - disableSELinux bool, savedConfigs *SavedConfigs, outputImageBase string) error { - - inputContentString, err := file.Read(isoGrubCfgFileName) - if err != nil { - return err - } - +func updateGrubCfgForLiveOS(initramfsImageType imagecustomizerapi.InitramfsImageType, inputContentString string, + disableSELinux bool, savedConfigs *SavedConfigs) (string, error) { searchCommand := fmt.Sprintf(searchCommandTemplate, isogenerator.DefaultVolumeId) - inputContentString, err = replaceSearchCommandAll(inputContentString, searchCommand) + inputContentString, err := replaceSearchCommandAll(inputContentString, searchCommand) if err != nil { - return fmt.Errorf("failed to update the search command in the iso grub.cfg:\n%w", err) + return "", fmt.Errorf("failed to update the search command in the iso grub.cfg:\n%w", err) } grubMkconfigEnabled := isGrubMkconfigConfig(inputContentString) @@ -48,121 +41,144 @@ func updateGrubCfg(isoGrubCfgFileName string, pxeGrubCfgFileName string, var oldLinuxPath string inputContentString, oldLinuxPath, err = setLinuxPath(inputContentString, isoKernelPath) if err != nil { - return fmt.Errorf("failed to update the kernel file path in the iso grub.cfg:\n%w", err) + return "", fmt.Errorf("failed to update the kernel file path in the iso grub.cfg:\n%w", err) } inputContentString, err = replaceToken(inputContentString, oldLinuxPath, isoKernelPath) if err != nil { - return fmt.Errorf("failed to update all the kernel file path occurances in the iso grub.cfg:\n%w", err) + return "", fmt.Errorf("failed to update all the kernel file path occurances in the iso grub.cfg:\n%w", err) } var oldInitrdPath string inputContentString, oldInitrdPath, err = setInitrdPath(inputContentString, isoInitrdPath) if err != nil { - return fmt.Errorf("failed to update the initrd file path in the iso grub.cfg:\n%w", err) + return "", fmt.Errorf("failed to update the initrd file path in the iso grub.cfg:\n%w", err) } inputContentString, err = replaceToken(inputContentString, oldInitrdPath, isoInitrdPath) if err != nil { - return fmt.Errorf("failed to update all the initrd file path occurances in the iso grub.cfg:\n%w", err) + return "", fmt.Errorf("failed to update all the initrd file path occurances in the iso grub.cfg:\n%w", err) } } else { inputContentString, _, err = setLinuxOrInitrdPathAll(inputContentString, linuxCommand, isoKernelPath, true /*allowMultiple*/) if err != nil { - return fmt.Errorf("failed to update the kernel file path in the iso grub.cfg:\n%w", err) + return "", fmt.Errorf("failed to update the kernel file path in the iso grub.cfg:\n%w", err) } inputContentString, _, err = setLinuxOrInitrdPathAll(inputContentString, initrdCommand, isoInitrdPath, true /*allowMultiple*/) if err != nil { - return fmt.Errorf("failed to update the initrd file path in the iso grub.cfg:\n%w", err) + return "", fmt.Errorf("failed to update the initrd file path in the iso grub.cfg:\n%w", err) } } - rootValue := fmt.Sprintf(rootValueLiveOSTemplate, isogenerator.DefaultVolumeId) - inputContentString, err = replaceKernelCommandLineArgValueAll(inputContentString, "root", rootValue) - if err != nil { - return fmt.Errorf("failed to update the root kernel argument in the iso grub.cfg:\n%w", err) + liveosKernelArgs := "" + switch initramfsImageType { + case imagecustomizerapi.InitramfsImageTypeFullOS: + argsToRemove := []string{"root"} + newArgs := []string{} + inputContentString, err = updateKernelCommandLineArgsAll(inputContentString, argsToRemove, newArgs) + if err != nil { + return "", fmt.Errorf("failed to update the root kernel argument in the iso grub.cfg:\n%w", err) + } + case imagecustomizerapi.InitramfsImageTypeBootstrap: + rootValue := fmt.Sprintf(rootValueLiveOSTemplate, isogenerator.DefaultVolumeId) + argsToRemove := []string{"root"} + newArgs := []string{"root=" + rootValue} + inputContentString, err = updateKernelCommandLineArgsAll(inputContentString, argsToRemove, newArgs) + if err != nil { + return "", fmt.Errorf("failed to update the root kernel argument in the iso grub.cfg:\n%w", err) + } + liveosKernelArgs = fmt.Sprintf(kernelArgsLiveOSTemplate, liveOSDir, liveOSImage) + default: + return "", fmt.Errorf("unsupported initramfs image type (%s)", initramfsImageType) } if disableSELinux { inputContentString, err = updateSELinuxCommandLineHelperAll(inputContentString, imagecustomizerapi.SELinuxModeDisabled) if err != nil { - return fmt.Errorf("failed to set SELinux mode:\n%w", err) + return "", fmt.Errorf("failed to set SELinux mode:\n%w", err) } } - liveosKernelArgs := fmt.Sprintf(kernelArgsLiveOSTemplate, liveOSDir, liveOSImage) savedArgs := GrubArgsToString(savedConfigs.Iso.KernelCommandLine.ExtraCommandLine) additionalKernelCommandline := liveosKernelArgs + " " + savedArgs inputContentString, err = appendKernelCommandLineArgsAll(inputContentString, additionalKernelCommandline) if err != nil { - return fmt.Errorf("failed to update the kernel arguments with the LiveOS configuration and user configuration in the iso grub.cfg:\n%w", err) + return "", fmt.Errorf("failed to update the kernel arguments with the LiveOS configuration and user configuration in the iso grub.cfg:\n%w", err) } - err = file.Write(inputContentString, isoGrubCfgFileName) + return inputContentString, nil +} + +func updateGrubCfgForPxe(initramfsImageType imagecustomizerapi.InitramfsImageType, inputContentString string, bootstrapBaseUrl string, + bootstrapFileUrl string) (string, error) { + // remove 'search' commands from PXE grub.cfg because it is not needed. + inputContentString, err := removeCommandAll(inputContentString, "search") if err != nil { - return fmt.Errorf("failed to write %s:\n%w", isoGrubCfgFileName, err) + return "", fmt.Errorf("failed to remove the 'search' commands from PXE grub.cfg:\n%w", err) } - // Check if the dracut version in use meets our minimum requirements for - // PXE support. - err = verifyDracutPXESupport(savedConfigs.OS.DracutPackageInfo) - if err != nil { - // MIC does not provide a way for the user to explicitly indicate that a - // PXE bootable ISO is desired. Instead, MIC always tries to create one. - // In cases that the source image does not meet the minimum requirements - // for the PXE bootable ISO, MIC just reports that information to the user - // and does not terminate the ISO creation process. No error is reported - // because MIC does not know if the user is interested only in the ISO image, - // or also in the PXE artifacts. - logger.Log.Infof("cannot generate grub.cfg for PXE booting.\n%v", err) - } else { - err = generatePxeGrubCfg(inputContentString, savedConfigs.Pxe.IsoImageBaseUrl, savedConfigs.Pxe.IsoImageFileUrl, - outputImageBase, pxeGrubCfgFileName) + if initramfsImageType == imagecustomizerapi.InitramfsImageTypeBootstrap { + bootstrapFileUrl, err = getPxeBootstrapFileUrl(bootstrapBaseUrl, bootstrapFileUrl) if err != nil { - return fmt.Errorf("failed to create grub configuration for PXE booting.\n%w", err) + return "", err + } + + rootValue := fmt.Sprintf(rootValuePxeTemplate, bootstrapFileUrl) + inputContentString, err = replaceKernelCommandLineArgValueAll(inputContentString, "root", rootValue) + if err != nil { + return "", fmt.Errorf("failed to update the root kernel argument with the PXE iso image url in the PXE grub.cfg:\n%w", err) + } + inputContentString, err = appendKernelCommandLineArgsAll(inputContentString, pxeKernelsArgs) + if err != nil { + return "", fmt.Errorf("failed to append the kernel arguments (%s) in the PXE grub.cfg:\n%w", pxeKernelsArgs, err) } } - return nil + return inputContentString, nil } -func generatePxeGrubCfg(inputContentString string, pxeIsoImageBaseUrl string, pxeIsoImageFileUrl string, - outputImageBase string, pxeGrubCfgFileName string) error { - if pxeIsoImageBaseUrl != "" && pxeIsoImageFileUrl != "" { - return fmt.Errorf("cannot set both iso image base url and full image url at the same time") - } +func updateGrubCfg(outputFormat imagecustomizerapi.ImageFormatType, initramfsImageType imagecustomizerapi.InitramfsImageType, + isoGrubCfgFileName string, disableSELinux bool, savedConfigs *SavedConfigs) error { + logger.Log.Infof("Updating ISO grub.cfg") - // remove 'search' commands from PXE grub.cfg because it is not needed. - inputContentString, err := removeCommandAll(inputContentString, "search") + inputContentString, err := file.Read(isoGrubCfgFileName) if err != nil { - return fmt.Errorf("failed to remove the 'search' commands from PXE grub.cfg:\n%w", err) + return err } - // If the specified URL is not a full path to an iso, append the generated - // iso file name to it. - if pxeIsoImageFileUrl == "" { - pxeIsoImageFileUrl, err = url.JoinPath(pxeIsoImageBaseUrl, outputImageBase) - if err != nil { - return fmt.Errorf("failed to concatenate URL (%s) and (%s)\n%w", pxeIsoImageBaseUrl, outputImageBase, err) - } - } - rootValue := fmt.Sprintf(rootValuePxeTemplate, pxeIsoImageFileUrl) - inputContentString, err = replaceKernelCommandLineArgValueAll(inputContentString, "root", rootValue) + inputContentString, err = updateGrubCfgForLiveOS(initramfsImageType, inputContentString, disableSELinux, savedConfigs) if err != nil { - return fmt.Errorf("failed to update the root kernel argument with the PXE iso image url in the PXE grub.cfg:\n%w", err) + return err } - inputContentString, err = appendKernelCommandLineArgsAll(inputContentString, pxeKernelsArgs) - if err != nil { - return fmt.Errorf("failed to append the kernel arguments (%s) in the PXE grub.cfg:\n%w", pxeKernelsArgs, err) + if outputFormat == imagecustomizerapi.ImageFormatTypePxe { + // Check if the dracut version in use meets our minimum requirements for + // PXE support. + err = verifyDracutPXESupport(savedConfigs.OS.DracutPackageInfo) + if err != nil { + // MIC does not provide a way for the user to explicitly indicate that a + // PXE bootable ISO is desired. Instead, MIC always tries to create one. + // In cases that the source image does not meet the minimum requirements + // for the PXE bootable ISO, MIC just reports that information to the user + // and does not terminate the ISO creation process. No error is reported + // because MIC does not know if the user is interested only in the ISO image, + // or also in the PXE artifacts. + logger.Log.Infof("cannot generate grub.cfg for PXE booting.\n%v", err) + } else { + inputContentString, err = updateGrubCfgForPxe(initramfsImageType, inputContentString, savedConfigs.Pxe.bootstrapBaseUrl, + savedConfigs.Pxe.bootstrapFileUrl) + if err != nil { + return fmt.Errorf("failed to create grub configuration for PXE booting.\n%w", err) + } + } } - err = file.Write(inputContentString, pxeGrubCfgFileName) + err = file.Write(inputContentString, isoGrubCfgFileName) if err != nil { - return fmt.Errorf("failed to write %s:\n%w", pxeGrubCfgFileName, err) + return fmt.Errorf("failed to write %s:\n%w", isoGrubCfgFileName, err) } return nil diff --git a/toolkit/tools/pkg/imagecustomizerlib/liveosisoimages.go b/toolkit/tools/pkg/imagecustomizerlib/liveosisoimages.go index dda1d54be6..392bfab9e2 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/liveosisoimages.go +++ b/toolkit/tools/pkg/imagecustomizerlib/liveosisoimages.go @@ -14,6 +14,7 @@ import ( "github.com/microsoft/azurelinux/toolkit/tools/imagecustomizerapi" "github.com/microsoft/azurelinux/toolkit/tools/imagegen/diskutils" "github.com/microsoft/azurelinux/toolkit/tools/internal/file" + "github.com/microsoft/azurelinux/toolkit/tools/internal/initrdutils" "github.com/microsoft/azurelinux/toolkit/tools/internal/isogenerator" "github.com/microsoft/azurelinux/toolkit/tools/internal/logger" "github.com/microsoft/azurelinux/toolkit/tools/internal/safechroot" @@ -24,6 +25,15 @@ import ( ) const ( + + // ToDo: this is not being invoked... + initScriptFileName = "init" + // Having #!/bin/bash header causes a kernel panic. + // Not having the init file at all, causes a kernel panic. + initContent = `mount -t proc proc /proc +/lib/systemd/systemd +` + dracutConfig = `add_dracutmodules+=" dmsquash-live livenet selinux " add_drivers+=" overlay " hostonly="no" @@ -39,22 +49,57 @@ hostonly="no" usrLibLocaleDir = "/usr/lib/locale" ) -func createInitrdImage(writeableRootfsDir, kernelVersion, outputInitrdPath string) error { - logger.Log.Debugf("Generating initrd (%s) from (%s)", outputInitrdPath, writeableRootfsDir) - - fstabFile := filepath.Join(writeableRootfsDir, "/etc/fstab") +func cleanFullOSFolderForLiveOS(fullOSDir string) error { + fstabFile := filepath.Join(fullOSDir, "/etc/fstab") logger.Log.Debugf("Deleting fstab from %s", fstabFile) + err := os.Remove(fstabFile) if err != nil { return fmt.Errorf("failed to delete fstab:\n%w", err) } - targetConfigFile := filepath.Join(writeableRootfsDir, "/etc/dracut.conf.d/20-live-cd.conf") - err = file.Write(dracutConfig, targetConfigFile) + logger.Log.Debugf("Deleting /boot") + err = os.RemoveAll(filepath.Join(fullOSDir, "boot")) + if err != nil { + return fmt.Errorf("failed to remove the /boot folder from the source image:\n%w", err) + } + + return nil +} + +func createFullOSInitrdImage(writeableRootfsDir, outputInitrdPath string) error { + logger.Log.Infof("Creating full OS initrd (%s) from (%s)", outputInitrdPath, writeableRootfsDir) + + err := cleanFullOSFolderForLiveOS(writeableRootfsDir) + if err != nil { + return fmt.Errorf("failed to clean root filesystem directory (%s):\n%w", writeableRootfsDir, err) + } + + // ToDo: Do we really need this?! + initScriptPath := filepath.Join(writeableRootfsDir, initScriptFileName) + err = os.WriteFile(initScriptPath, []byte(initContent), 0755) if err != nil { - return fmt.Errorf("failed to create %s:\n%w", targetConfigFile, err) + return fmt.Errorf("failed to create (%s):\n%w", initScriptPath, err) } + err = initrdutils.CreateInitrdImageFromFolder(writeableRootfsDir, outputInitrdPath) + if err != nil { + return fmt.Errorf("failed to create the initrd image:\n%w", err) + } + + return nil +} + +func createBootstrapInitrdImage(writeableRootfsDir, kernelVersion, outputInitrdPath string) error { + logger.Log.Infof("Creating bootstrap initrd (%s) from (%s)", outputInitrdPath, writeableRootfsDir) + + dracutConfigFile := filepath.Join(writeableRootfsDir, "/etc/dracut.conf.d/20-live-cd.conf") + err := file.Write(dracutConfig, dracutConfigFile) + if err != nil { + return fmt.Errorf("failed to create %s:\n%w", dracutConfigFile, err) + } + defer os.Remove(dracutConfigFile) + chroot := safechroot.NewChroot(writeableRootfsDir, true /*isExistingDir*/) if chroot == nil { return fmt.Errorf("failed to create a new chroot object for %s.", writeableRootfsDir) @@ -98,11 +143,11 @@ func createInitrdImage(writeableRootfsDir, kernelVersion, outputInitrdPath strin } func createSquashfsImage(writeableRootfsDir, outputSquashfsPath string) error { - logger.Log.Debugf("Creating squashfs (%s) from (%s)", outputSquashfsPath, writeableRootfsDir) + logger.Log.Infof("Creating squashfs (%s) from (%s)", outputSquashfsPath, writeableRootfsDir) - err := os.RemoveAll(filepath.Join(writeableRootfsDir, "boot")) + err := cleanFullOSFolderForLiveOS(writeableRootfsDir) if err != nil { - return fmt.Errorf("failed to remove the /boot folder from the source image:\n%w", err) + return fmt.Errorf("failed to clean root filesystem directory (%s):\n%w", writeableRootfsDir, err) } exists, err := file.PathExists(outputSquashfsPath) @@ -154,32 +199,31 @@ func stageIsoFiles(filesStore *IsoFilesStore, baseConfigPath string, additionalI // map of file full local path to location on iso media. artifactsToIsoMap := map[string]string{ - filesStore.isoBootImagePath: "boot/grub2", - filesStore.isoGrubCfgPath: "boot/grub2", - filesStore.vmlinuzPath: "boot", - filesStore.initrdImagePath: "boot", - filesStore.squashfsImagePath: "liveos", + filesStore.isoBootImagePath: "boot/grub2", + filesStore.isoGrubCfgPath: "boot/grub2", + filesStore.vmlinuzPath: "boot", + filesStore.initrdImagePath: "boot", } - // Add optional saved config file if it exists. - if filesStore.savedConfigsFilePath != "" { - exists, err := file.PathExists(filesStore.savedConfigsFilePath) + // Add optional squashfs file if it exists. + if filesStore.squashfsImagePath != "" { + exists, err := file.PathExists(filesStore.squashfsImagePath) if err != nil { - return fmt.Errorf("failed to check if (%s) exists:\n%w", filesStore.savedConfigsFilePath, err) + return fmt.Errorf("failed to check if (%s) exists:\n%w", filesStore.squashfsImagePath, err) } if exists { - artifactsToIsoMap[filesStore.savedConfigsFilePath] = "azl-image-customizer" + artifactsToIsoMap[filesStore.squashfsImagePath] = "liveos" } } - // Add optional grub-pxe.cfg file if it exists. - if filesStore.pxeGrubCfgPath != "" { - exists, err := file.PathExists(filesStore.pxeGrubCfgPath) + // Add optional saved config file if it exists. + if filesStore.savedConfigsFilePath != "" { + exists, err := file.PathExists(filesStore.savedConfigsFilePath) if err != nil { - return fmt.Errorf("failed to check if (%s) exists:\n%w", filesStore.pxeGrubCfgPath, err) + return fmt.Errorf("failed to check if (%s) exists:\n%w", filesStore.savedConfigsFilePath, err) } if exists { - artifactsToIsoMap[filesStore.pxeGrubCfgPath] = "boot/grub2" + artifactsToIsoMap[filesStore.savedConfigsFilePath] = "azl-image-customizer" } } @@ -231,7 +275,7 @@ func stageIsoFiles(filesStore *IsoFilesStore, baseConfigPath string, additionalI return nil } -func createIsoImage(buildDir string, filesStore *IsoFilesStore, baseConfigPath string, +func createIsoImage(buildDir string, baseConfigPath string, filesStore *IsoFilesStore, additionalIsoFiles imagecustomizerapi.AdditionalFileList, outputImagePath string) error { stagingDir := filepath.Join(buildDir, "staging") @@ -290,43 +334,60 @@ func getDiskSizeEstimateInMBs(filesOrDirs []string, safetyFactor float64) (size return estimatedSizeInMBs, nil } -func createWriteableImageFromArtifacts(buildDir string, filesStore *IsoFilesStore, rawImageFile string) error { - - logger.Log.Infof("Creating writeable image from squashfs (%s)", filesStore.squashfsImagePath) +func createWriteableImageFromArtifacts(buildDir string, artifactsStore *IsoArtifactsStore, rawImageFile string) error { + logger.Log.Infof("Creating full OS writeable image from ISO artifacts") - // rootfs folder (mount squash fs) - squashMountDir, err := os.MkdirTemp(buildDir, "tmp-squashfs-mount-") + rootfsDir, err := os.MkdirTemp(buildDir, "tmp-full-os-root-") if err != nil { return fmt.Errorf("failed to create temporary mount folder for squashfs:\n%w", err) } - defer os.RemoveAll(squashMountDir) + defer os.RemoveAll(rootfsDir) - squashfsLoopDevice, err := safeloopback.NewLoopback(filesStore.squashfsImagePath) + squashfsExists, err := file.PathExists(artifactsStore.files.squashfsImagePath) if err != nil { - return fmt.Errorf("failed to create loop device for (%s):\n%w", filesStore.squashfsImagePath, err) + return fmt.Errorf("failed to check if the squash root file system image exists (%s):\n%w", artifactsStore.files.squashfsImagePath, err) } - defer squashfsLoopDevice.Close() - squashfsMount, err := safemount.NewMount(squashfsLoopDevice.DevicePath(), squashMountDir, - "squashfs" /*fstype*/, 0 /*flags*/, "" /*data*/, false /*makeAndDelete*/) - if err != nil { - return err + var squashfsLoopDevice *safeloopback.Loopback + var squashfsMount *safemount.Mount + + if squashfsExists { + logger.Log.Infof("Detected bootstrap OS initrd configuration") + squashfsLoopDevice, err = safeloopback.NewLoopback(artifactsStore.files.squashfsImagePath) + if err != nil { + return fmt.Errorf("failed to create loop device for (%s):\n%w", artifactsStore.files.squashfsImagePath, err) + } + defer squashfsLoopDevice.Close() + + squashfsMount, err = safemount.NewMount(squashfsLoopDevice.DevicePath(), rootfsDir, + "squashfs" /*fstype*/, 0 /*flags*/, "" /*data*/, false /*makeAndDelete*/) + if err != nil { + return err + } + defer squashfsMount.Close() + } else { + logger.Log.Infof("Detected full OS initrd configuration") + err = initrdutils.CreateFolderFromInitrdImage(artifactsStore.files.initrdImagePath, rootfsDir) + if err != nil { + return fmt.Errorf("failed to extract files from the initrd image (%s):\n%w", artifactsStore.files.initrdImagePath, err) + } } - defer squashfsMount.Close() + + logger.Log.Infof("Populated (%s) with full file system", rootfsDir) // boot folder (from artifacts) - artifactsBootDir := filepath.Join(filesStore.artifactsDir, "boot") + artifactsBootDir := filepath.Join(artifactsStore.files.artifactsDir, "boot") imageContentList := []string{ - squashMountDir, - filesStore.bootEfiPath, - filesStore.grubEfiPath, + rootfsDir, + artifactsStore.files.bootEfiPath, + artifactsStore.files.grubEfiPath, artifactsBootDir} // estimate the new disk size safeDiskSizeMB, err := getDiskSizeEstimateInMBs(imageContentList, expansionSafetyFactor) if err != nil { - return fmt.Errorf("failed to calculate the disk size of %s:\n%w", squashMountDir, err) + return fmt.Errorf("failed to calculate the disk size of %s:\n%w", rootfsDir, err) } logger.Log.Debugf("safeDiskSizeMB = %d", safeDiskSizeMB) @@ -373,18 +434,19 @@ func createWriteableImageFromArtifacts(buildDir string, filesStore *IsoFilesStor }, } - targetOs, err := targetos.GetInstalledTargetOs(squashMountDir) + targetOs, err := targetos.GetInstalledTargetOs(rootfsDir) if err != nil { return fmt.Errorf("failed to determine target OS of ISO squashfs:\n%w", err) } // populate the newly created disk image with content from the squash fs installOSFunc := func(imageChroot *safechroot.Chroot) error { + logger.Log.Infof("Installing files to empty image") // At the point when this copy will be executed, both the boot and the // root partitions will be mounted, and the files of /boot/efi will // land on the the boot partition, while the rest will be on the rootfs // partition. - err := copyPartitionFiles(squashMountDir+"/.", imageChroot.RootDir()) + err := copyPartitionFiles(rootfsDir+"/.", imageChroot.RootDir()) if err != nil { return fmt.Errorf("failed to copy squashfs contents to a writeable disk:\n%w", err) } @@ -396,28 +458,49 @@ func createWriteableImageFromArtifacts(buildDir string, filesStore *IsoFilesStor // pull the boot artifacts back into the full file system so that // it is restored to its original state and subsequent customization // or extraction can proceed transparently. - err = copyPartitionFiles(artifactsBootDir, imageChroot.RootDir()) if err != nil { return fmt.Errorf("failed to copy (%s) contents to a writeable disk:\n%w", artifactsBootDir, err) } + // The `initrd.img` must be on the form `initrd-*` so that `grub2-mkconfig` + // can find it. If it cannot find it, the generated grub.cfg will be missing + // all the boot entries. + initrdFileName := fmt.Sprintf("initrd-%s.img", artifactsStore.info.kernelVersion) + initrdOld := filepath.Join(imageChroot.RootDir(), "boot/initrd.img") + initrdNew := filepath.Join(imageChroot.RootDir(), "boot", initrdFileName) + err = os.Rename(initrdOld, initrdNew) + if err != nil { + return fmt.Errorf("failed to rename (%s) to (%s)", initrdOld, initrdNew) + } + + // The `vmlinuz` must be on the form `vmlinuz-*` so that `grub2-mkconfig` + // can find it. If it cannot find it, the generated grub.cfg will be missing + // all the boot entries. + kernelFileName := fmt.Sprintf("vmlinuz-%s", artifactsStore.info.kernelVersion) + kernelOld := filepath.Join(imageChroot.RootDir(), "boot/vmlinuz") + kernelNew := filepath.Join(imageChroot.RootDir(), "boot", kernelFileName) + err = os.Rename(kernelOld, kernelNew) + if err != nil { + return fmt.Errorf("failed to rename (%s) to (%s)", kernelOld, kernelNew) + } + targetEfiDir := filepath.Join(imageChroot.RootDir(), "boot/efi/EFI/BOOT") err = os.MkdirAll(targetEfiDir, os.ModePerm) if err != nil { return fmt.Errorf("failed to create destination efi directory (%s):\n%w", targetEfiDir, err) } - targetShimPath := filepath.Join(targetEfiDir, filepath.Base(filesStore.bootEfiPath)) - err = file.Copy(filesStore.bootEfiPath, targetShimPath) + targetShimPath := filepath.Join(targetEfiDir, filepath.Base(artifactsStore.files.bootEfiPath)) + err = file.Copy(artifactsStore.files.bootEfiPath, targetShimPath) if err != nil { - return fmt.Errorf("failed to copy (%s) to (%s):\n%w", filesStore.bootEfiPath, targetShimPath, err) + return fmt.Errorf("failed to copy (%s) to (%s):\n%w", artifactsStore.files.bootEfiPath, targetShimPath, err) } - targetGrubPath := filepath.Join(targetEfiDir, filepath.Base(filesStore.grubEfiPath)) - err = file.Copy(filesStore.grubEfiPath, targetGrubPath) + targetGrubPath := filepath.Join(targetEfiDir, filepath.Base(artifactsStore.files.grubEfiPath)) + err = file.Copy(artifactsStore.files.grubEfiPath, targetGrubPath) if err != nil { - return fmt.Errorf("failed to copy (%s) to (%s):\n%w", filesStore.grubEfiPath, targetGrubPath, err) + return fmt.Errorf("failed to copy (%s) to (%s):\n%w", artifactsStore.files.grubEfiPath, targetGrubPath, err) } return err @@ -431,14 +514,18 @@ func createWriteableImageFromArtifacts(buildDir string, filesStore *IsoFilesStor return fmt.Errorf("failed to copy squashfs into new writeable image (%s):\n%w", rawImageFile, err) } - err = squashfsMount.CleanClose() - if err != nil { - return err + if squashfsMount != nil { + err = squashfsMount.CleanClose() + if err != nil { + return err + } } - err = squashfsLoopDevice.CleanClose() - if err != nil { - return err + if squashfsLoopDevice != nil { + err = squashfsLoopDevice.CleanClose() + if err != nil { + return err + } } return nil diff --git a/toolkit/tools/pkg/imagecustomizerlib/liveosisoutils.go b/toolkit/tools/pkg/imagecustomizerlib/liveosisoutils.go index 4f0a8f5f7c..68858f5d26 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/liveosisoutils.go +++ b/toolkit/tools/pkg/imagecustomizerlib/liveosisoutils.go @@ -32,8 +32,6 @@ const ( isoGrubCfg = "grub.cfg" isoGrubCfgPath = grubCfgDir + "/" + isoGrubCfg - pxeGrubCfg = "grub-pxe.cfg" - isoBootDir = "boot" initrdImage = "initrd.img" @@ -207,7 +205,7 @@ func extractIsoImageContents(buildDir string, isoImageFile string, isoExpansionF err = os.MkdirAll(isoExpansionFolder, os.ModePerm) if err != nil { - return fmt.Errorf("failed to create folder %s:\n%w", isoExpansionFolder, err) + return fmt.Errorf("failed to create folder (%s):\n%w", isoExpansionFolder, err) } err = copyPartitionFiles(mountDir+"/.", isoExpansionFolder) diff --git a/toolkit/tools/pkg/imagecustomizerlib/main_test.go b/toolkit/tools/pkg/imagecustomizerlib/main_test.go index 3a55952747..9cdfc92f0c 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/main_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/main_test.go @@ -31,7 +31,7 @@ const ( // Most features don't have version Azure Linux version specific behavior. // So, there is only minimal value in duplicating the tests across versions for such features. - baseImageVersionDefault = baseImageVersionAzl2 + baseImageVersionDefault = baseImageVersionAzl3 ) var ( diff --git a/toolkit/tools/pkg/imagecustomizerlib/resolvconf_test.go b/toolkit/tools/pkg/imagecustomizerlib/resolvconf_test.go index 9ba9423aaa..842002b457 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/resolvconf_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/resolvconf_test.go @@ -30,7 +30,7 @@ func TestCustomizeImageResolvConfDelete(t *testing.T) { } err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -93,7 +93,7 @@ func TestCustomizeImageResolvConfRestoreFile(t *testing.T) { } err = CustomizeImage(buildDir, testDir, &config, outImageFilePath, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -163,7 +163,7 @@ func TestCustomizeImageResolvConfRestoreSymlink(t *testing.T) { } err = CustomizeImage(buildDir, testDir, &config, outImageFilePath, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } @@ -201,7 +201,7 @@ func TestCustomizeImageResolvConfNewSymlink(t *testing.T) { } err := CustomizeImage(buildDir, testDir, &config, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/runscripts_test.go b/toolkit/tools/pkg/imagecustomizerlib/runscripts_test.go index c1f2ab9433..7fc35d55ba 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/runscripts_test.go +++ b/toolkit/tools/pkg/imagecustomizerlib/runscripts_test.go @@ -23,7 +23,7 @@ func TestCustomizeImageRunScripts(t *testing.T) { // Customize image. err = CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", - "" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) + false /*useBaseImageRpmRepos*/, "" /*packageSnapshotTime*/) if !assert.NoError(t, err) { return } diff --git a/toolkit/tools/pkg/imagecustomizerlib/savedconfigs.go b/toolkit/tools/pkg/imagecustomizerlib/savedconfigs.go index b56a1c0b50..4902b59aed 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/savedconfigs.go +++ b/toolkit/tools/pkg/imagecustomizerlib/savedconfigs.go @@ -10,6 +10,7 @@ import ( "github.com/microsoft/azurelinux/toolkit/tools/imagecustomizerapi" "github.com/microsoft/azurelinux/toolkit/tools/internal/file" + "github.com/microsoft/azurelinux/toolkit/tools/internal/logger" ) // 'SavedConfigs' is a subset of the Image Customizer input configurations that @@ -41,19 +42,19 @@ func (i *IsoSavedConfigs) IsValid() error { } type PxeSavedConfigs struct { - IsoImageBaseUrl string `yaml:"isoImageBaseUrl"` - IsoImageFileUrl string `yaml:"isoImageFileUrl"` + bootstrapBaseUrl string `yaml:"bootstrapBaseUrl"` + bootstrapFileUrl string `yaml:"bootstrapFileUrl"` } func (p *PxeSavedConfigs) IsValid() error { - if p.IsoImageBaseUrl != "" && p.IsoImageFileUrl != "" { - return fmt.Errorf("cannot specify both 'isoImageBaseUrl' and 'isoImageFileUrl' at the same time.") + if p.bootstrapBaseUrl != "" && p.bootstrapFileUrl != "" { + return fmt.Errorf("cannot specify both 'bootstrapBaseUrl' and 'bootstrapFileUrl' at the same time.") } - err := imagecustomizerapi.IsValidPxeUrl(p.IsoImageBaseUrl) + err := imagecustomizerapi.IsValidPxeUrl(p.bootstrapBaseUrl) if err != nil { return err } - err = imagecustomizerapi.IsValidPxeUrl(p.IsoImageFileUrl) + err = imagecustomizerapi.IsValidPxeUrl(p.bootstrapFileUrl) if err != nil { return err } @@ -61,6 +62,7 @@ func (p *PxeSavedConfigs) IsValid() error { } type OSSavedConfigs struct { + KernelVersion string `yaml:"kernelVersion"` DracutPackageInfo *PackageVersionInformation `yaml:"dracutPackage"` RequestedSELinuxMode imagecustomizerapi.SELinuxMode `yaml:"selinuxRequestedMode"` SELinuxPolicyPackageInfo *PackageVersionInformation `yaml:"selinuxPolicyPackage"` @@ -128,14 +130,16 @@ func loadSavedConfigs(savedConfigsFilePath string) (savedConfigs *SavedConfigs, return savedConfigs, nil } -func updateSavedConfigs(savedConfigsFilePath string, newKernelArgs []string, - newPxeIsoImageBaseUrl string, newPxeIsoImageFileUrl string, newDracutPackageInfo *PackageVersionInformation, +func updateSavedConfigs(savedConfigsFilePath string, newKernelCommandLine imagecustomizerapi.KernelCommandLine, + newBootstrapBaseUrl string, newBootstrapFileUrl string, newKernelVersion string, newDracutPackageInfo *PackageVersionInformation, newRequestedSelinuxMode imagecustomizerapi.SELinuxMode, newSELinuxPackageInfo *PackageVersionInformation, ) (outputConfigs *SavedConfigs, err error) { + logger.Log.Infof("Updating saved configurations") outputConfigs = &SavedConfigs{} - outputConfigs.Iso.KernelCommandLine.ExtraCommandLine = newKernelArgs - outputConfigs.Pxe.IsoImageBaseUrl = newPxeIsoImageBaseUrl - outputConfigs.Pxe.IsoImageFileUrl = newPxeIsoImageFileUrl + outputConfigs.Iso.KernelCommandLine = newKernelCommandLine + outputConfigs.Pxe.bootstrapBaseUrl = newBootstrapBaseUrl + outputConfigs.Pxe.bootstrapFileUrl = newBootstrapFileUrl + outputConfigs.OS.KernelVersion = newKernelVersion outputConfigs.OS.DracutPackageInfo = newDracutPackageInfo outputConfigs.OS.RequestedSELinuxMode = newRequestedSelinuxMode outputConfigs.OS.SELinuxPolicyPackageInfo = newSELinuxPackageInfo @@ -150,7 +154,7 @@ func updateSavedConfigs(savedConfigsFilePath string, newKernelArgs []string, if len(inputConfigs.Iso.KernelCommandLine.ExtraCommandLine) > 0 { // If yes, add them before the new kernel arguments. savedArgs := inputConfigs.Iso.KernelCommandLine.ExtraCommandLine - newArgs := newKernelArgs + newArgs := newKernelCommandLine.ExtraCommandLine // Combine saved arguments with new ones combinedArgs := append(savedArgs, newArgs...) @@ -158,23 +162,27 @@ func updateSavedConfigs(savedConfigsFilePath string, newKernelArgs []string, } // if the PXE iso image url is not set, set it to the value from the previous run. - if newPxeIsoImageBaseUrl == "" && inputConfigs.Pxe.IsoImageBaseUrl != "" { - outputConfigs.Pxe.IsoImageBaseUrl = inputConfigs.Pxe.IsoImageBaseUrl + if newBootstrapBaseUrl == "" && inputConfigs.Pxe.bootstrapBaseUrl != "" { + outputConfigs.Pxe.bootstrapBaseUrl = inputConfigs.Pxe.bootstrapBaseUrl } - if newPxeIsoImageFileUrl == "" && inputConfigs.Pxe.IsoImageFileUrl != "" { - outputConfigs.Pxe.IsoImageFileUrl = inputConfigs.Pxe.IsoImageFileUrl + if newBootstrapFileUrl == "" && inputConfigs.Pxe.bootstrapFileUrl != "" { + outputConfigs.Pxe.bootstrapFileUrl = inputConfigs.Pxe.bootstrapFileUrl } - // if IsoImageBaseUrl is being set in this run (i.e. newPxeIsoImageBaseUrl != ""), - // then make sure IsoImageFileUrl is unset (since both fields must be mutually + // if bootstrapBaseUrl is being set in this run (i.e. newBootstrapBaseUrl != ""), + // then make sure bootstrapFileUrl is unset (since both fields must be mutually // exclusive) - and vice versa. - if newPxeIsoImageBaseUrl != "" { - outputConfigs.Pxe.IsoImageFileUrl = "" + if newBootstrapBaseUrl != "" { + outputConfigs.Pxe.bootstrapFileUrl = "" } - if newPxeIsoImageFileUrl != "" { - outputConfigs.Pxe.IsoImageBaseUrl = "" + if newBootstrapFileUrl != "" { + outputConfigs.Pxe.bootstrapBaseUrl = "" + } + + if newKernelVersion == "" { + outputConfigs.OS.KernelVersion = inputConfigs.OS.KernelVersion } // newOSDracutVersion can be nil if the input is an ISO and the diff --git a/toolkit/tools/pkg/imagecustomizerlib/testdata/files/c.txt b/toolkit/tools/pkg/imagecustomizerlib/testdata/files/c.txt new file mode 100644 index 0000000000..3dd415dd3f --- /dev/null +++ b/toolkit/tools/pkg/imagecustomizerlib/testdata/files/c.txt @@ -0,0 +1 @@ +qrstuvwx diff --git a/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-os-vm-config.yaml b/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-bootstrap-vm.yaml similarity index 92% rename from toolkit/tools/pkg/imagecustomizerlib/testdata/iso-os-vm-config.yaml rename to toolkit/tools/pkg/imagecustomizerlib/testdata/iso-bootstrap-vm.yaml index 2b82667e55..669ab680cf 100644 --- a/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-os-vm-config.yaml +++ b/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-bootstrap-vm.yaml @@ -1,3 +1,5 @@ +iso: + initramfsType: full-os os: selinux: mode: disabled diff --git a/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-files-and-args-config.yaml b/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-files-and-args-config.yaml deleted file mode 100644 index a2f0869a93..0000000000 --- a/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-files-and-args-config.yaml +++ /dev/null @@ -1,18 +0,0 @@ -pxe: - isoImageBaseUrl: http://my-pxe-server-1 -iso: - additionalFiles: - - source: files/a.txt - destination: /a.txt - - kernelCommandLine: - extraCommandLine: - - rd.info - -os: - packages: - install: - - squashfs-tools - - tar - - device-mapper - - curl diff --git a/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-full-os-vm.yaml b/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-full-os-vm.yaml new file mode 100644 index 0000000000..2f81feb8d8 --- /dev/null +++ b/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-full-os-vm.yaml @@ -0,0 +1,20 @@ +iso: + initramfsType: bootstrap +os: + selinux: + mode: disabled + kernelCommandLine: + extraCommandLine: + - "selinux=0" + packages: + install: + # iso required packages + - squashfs-tools + - tar + - device-mapper + - curl + + additionalFiles: + # Enable DHCP client on all of the physical NICs. + - source: files/89-ethernet.network + destination: /etc/systemd/network/89-ethernet.network diff --git a/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-os-prereqs-config.yaml b/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-os-prereqs-config.yaml deleted file mode 100644 index 59c213a594..0000000000 --- a/toolkit/tools/pkg/imagecustomizerlib/testdata/iso-os-prereqs-config.yaml +++ /dev/null @@ -1,7 +0,0 @@ -os: - packages: - install: - - squashfs-tools - - tar - - device-mapper - - curl