Add a Swap section and support for .swap systemd units.
This commit also adds a separate filename regex for unit files that
differs from the one we have for general files.  Specifically, it allows
for backslashes, which we will need to create .swap unit files.  The
filenames for these units must be the escaped path to the device or file
they control, which can contain escape characters (e.g. \x2d for
escaping a dash).

The Service and Install sections are now marked 'omitempty' since they
can be omitted (depending on the type of the unit).

See also systemd.swap(5)

GenSystemdMountStages() generates systemd .mount and .swap stages based
on a partition table.  This is an alternative to generating an fstab
stage for the mountpoints.

The generator creates one systemd.unit.create stage for each mountpoint
(and swap) and an extra stage that enables all the new units after they
are written.

vfat volume IDs are discovered by matching against a new regular
expression defined in the `mkfs_fat_stage.go` file.  This is needed
because non-UUID identifiers are not lowercased in /dev/disk/by-uuid.

Test GenSystemdMountStages() using test partition tables from testdisk.

Add a boolean to each pipeline that generates a filesystem description
(i.e. creates an org.osbuild.fstab stage), which when enabled will
create systemd mount units instead of writing the static filesystem
description to /etc/fstab.

Test all pipelines that support filesystem configuration switching
between /etc/fstab and systemd .mount units.  These are:
- OS
- OSTreeDeployment
- RawBootcImage

This commit also includes some test restructuring.
- The OS pipeline tests have been moved to manifest_test, external to
  the manifest package.
- Functions that are needed for both internal and external tests are
  defined and exported from the export_test.go file in the manifest
  package.  We should move all tests to the external manifest_test
  package eventually.

The testing is a little inconsistent now between different kinds of
pipelines.  Some run serializeStart() and serialize() in the same
exported function, and others (bootc) export both.  These should be
made consistent, but for now it's okay.

Add a flag for enabling mount units on ImageConfig and copy the value to
the OSCustomizations (for rpm-based disk images) and the
OSTreeDploymentCustomizations.

See osbuild/bootc-image-builder#756

Add repos for branched (42) and rawhide (43) to start testing.
Generate the new gitlab config.

Make KernelOptions a string slice where each element is a separate
option.  This makes it easier to manipulate each option individually,
which we will need later to modify default options when using mount
units.

The options are defined in functions instead of global vars to avoid
modifying the static values.

This commit has a non-functional effect on manifest generation:
Edge and IoT images that use org.osbuild.ostree.deploy stage now define
each option as a separate element in the `kernel_opts` slice.
Previously, it would be a mix of space-separated multi-option strings
and individual ones.

Move the root UUID discovery from the OS pipeline into the
GenImageKernelOptions() function.  Previously this was done in the
prependKernelCmdlineStage().  That function now simply generates the
stage and adds it to the top of the pipeline and doesn't do any
discovery on its own.

The following commits will add more filesystem-related kernel
command-line argument handling to the osbuild function.

To be reused in new tests.

When /usr is on a separate mountpoint and we're not writing an
/etc/fstab, the filesystem information should be included in the kernel
cmdline arguments for autodiscovery to work.

The systemd-remount-fs service reads mount options from /etc/fstab for
the root partition (and /usr, if separate).  If there is no /etc/fstab,
the fs is not remounted, so a read-only mount will stay read-only on
boot.

Switch the "ro" command line argument to "rw" if we're not writing an
/etc/fstab file.
Also, add the root mount options as rootflags when they are not just
"defaults".

Add tests for the new functionality in GenImageKernelOptions().

Starting from Fedora 43, use mount units instead of writing /etc/fstab
for the minimal-raw image type.

Setting IMAGE_BUILDER_EXPERIMENTAL="no-fstab" will enable mount units
without /etc/fstab for any image type.

Rename it to avoid shadowing the built-in function.

Don't write out the locale file to disk when it is unset in the
OSCustomizations.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>

Moves the hostname to the image config.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>

Add a small test to see if hostname stages are added or not.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>

gen-manifests-diff is a helper to diff the generated manifests
against what the current "main" of the images library would do.

Alternatively a revision can be given as the first argument
and with that the diff is calculated against that. All images
types and architectures are used by default. It is still
very fast (typically 3s).

Example output:
```
calculating diff against d285ec6
found differences between /tmp/tmpbjhime7p/new and reference manifests:
diff -uNr /tmp/tmpbjhime7p/ref/centos_9-aarch64-edge_ami-edge_ostree_pull_user.json /tmp/tmpbjhime7p/new/centos_9-aarch64-edge_ami-edge_ostree_pull_user.json
--- /tmp/tmpbjhime7p/ref/centos_9-aarch64-edge_ami-edge_ostree_pull_user.json   2025-03-10 20:55:20.736477268 +0100
+++ /tmp/tmpbjhime7p/new/centos_9-aarch64-edge_ami-edge_ostree_pull_user.json   2025-03-10 20:55:22.294453956 +0100
@@ -161,7 +161,13 @@
             },
             "kernel_opts": [
               "luks.uuid=e3eb0343-83b0-441c-a8aa-6a715afc1d39",
-              "console=tty0 console=ttyS0,115200n8 net.ifnames=0 nvme_core.io_timeout=4294967295 modprobe.blacklist=vc4 rw coreos.no_persist_ip",
+              "console=tty0",
+              "console=ttyS0,115200n8",
+              "net.ifnames=0",
+              "nvme_core.io_timeout=4294967295",
+              "modprobe.blacklist=vc4",
+              "rw",
+              "coreos.no_persist_ip",
               "ignition.platform.id=metal",
               "$ignition_firstboot"
             ]
...
```

This is (hopefully) the uncontroversial part of
#1307
and already useful when checking that a yamlification (or other
change) is not resulting in any regressions. Also useful to see
if a change results in the expected change(s).

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>

This is a better version of #1312
that moves the de-duplication of packages into the package depsolver
of gen-manifests. This is similar to how osbuild-depsolve-dnf works
that also will present a de-duplicated result.

Thanks to thozza for the suggestion to do it this way.

This is used to setup the configuration file for an ISO that boots with
grub2 and BIOS.

The grub2 BIOS iso needs an eltorito.img built by grub2-mkimage. This is
created when location isn't passed to org.osbuild.grub2.inst and the
filesystem is set to iso9660.

In order to support this some of the fields become optional, and iso9660
is added to the list of allowed filesystems.

This adds an ISOBootType enum that replaces the ISOLinux bool in order
to control what type of boot iso is created:
 - UEFI only using grub2
 - syslinux on BIOS and grub2 UEFI
 - grub2 for both BIOS and UEFI

It also adds the Grub2MBR option to the xorrisofs stage.  Setting this
to the path of the grub2 hybrid boot image
(eg./usr/lib/grub/i386-pc/boot_hybrid.img) will cause the created ISO to
use grub2 to boot on BIOS systems.

This overrides the presence of the ISOHybridPath and does not use
syslinux.

In order to support this it does require that the grub2-pc-modules
package is installed.

This also adds a stage using Grub2InstStage to create the eltorito.img
needed by grub2.

On Fedora select Grub2ISOBoot when building on x86, otherwise the ISO is
UEFI only.

On Fedora select Grub2ISOBoot when building on x86, otherwise the ISO is
UEFI only.

On RHEL x86 builds use Grub2ISOBoot for RHEL10, SyslinuxISOBoot for
other versions and on non-x86 use UEFI only.

On Fedora select Grub2ISOBoot when building on x86, otherwise the ISO is
UEFI only.

On RHEL x86 builds use Grub2ISOBoot for RHEL10, SyslinuxISOBoot for
other versions and on non-x86 use UEFI only.

OSTree simplified always uses SyslinuxISOBoot.

As suggested by thozza this adds a note about the limits of
`gen-manifests-diff`.

Thanks to thozza.

This commit tweaks the packageSetFunc() to return an error. With
that we can avoid `panics()` and instead just pass the error up.

Thanks to Brian for suggesting to clean this up.

This commit adds a new `IMAGE_BUILDER_EXPERIMENTAL=yamldir=...`
experimental flag to force setting the datadir for the yaml
directory to override e.g. packagesets.

Ideally we would pass this all the way down the stack via a
search path but this will require a bunch of more refactoring.
So to give people the ability to tweak packagesets today, start
here.

Add a short explaination about local development for the yaml
definitions.

This commit adds a basic imagetype yaml to support merging
of package sets  This is a useful feature so that we can e.g.
avoid duplicating the `cloud_base` or `anaconda` package sets.

It works by adding new toplevels "image_types" in the
package_sets.yaml and then using anchors/refs to refer to
them in the image_type yaml. To allow sharing yaml that is
not part of an image_type it also introduces a top-level
`.common` that can have `any` yaml in it.

It allows us to write:
```yaml
.common:
  base: &base_pkgset
    include: [from-base-inc]
    exclude: [from-base-exc]
    condition:
      distro_name:
        test-distro:
          include: [from-base-condition-inc]
          exclude: [from-base-condition-exc]

image_types:
  other_type:
    package_sets:
      - &other_type_pkgset
        include: [from-other-type-inc]
        exclude: [from-other-type-exc]
  test_type:
    package_sets:
      - *base_pkgset
      - *other_type_pkgset
      - include: [from-type-inc]
        exclude: [from-type-exc]
        condition:
          distro_name:
            test-distro:
              include: [from-condition-inc]
              exclude: [from-condition-exc]
```
The downside of this approach is that:
1. it requires the anchor to be there before the ref, which can
   be confusing (we could solve this later via includes)
2. if the anchor is forgotten the error becomes very obscure
3. its a bit "hardcore" yaml

The upside is that it is very symetrical and actually feels quite
nice.

Add support for specifying the partition type for disk customizations.
This is the type of the partition, specified as a GUID (UUID) for GPT
partitions (see for example the Discoverable Partitions Specification
[1]) or a 2-digit hex number for DOS partitions [2].

Ideally we would call this parameter 'type' but we already use that for
defining 'plain', 'lvm', or 'btrfs'.

The validator checks the format of the type based on the partition table
type if set, otherwise checks for both because the fallback type is
unknown at the time.

[1] https://uapi-group.org/specifications/specs/discoverable_partitions_specification/
[2] https://tldp.org/HOWTO/Partition-Mass-Storage-Definitions-Naming-HOWTO/x190.html

Read the custom part type if it's set, otherwise infer it from the
mountpoint and partition table type.

At this stage, in NewCustomPartitionTable(), we know the partition table
type, so we can verify the partition type ID again with more strictness.

To avoid duplicating the validator, we use the partition method again.
It would have been nicer to have this kind of validation in the disk
package instead of the blueprint, but that would create an import cycle.

Tiny addition to the doc to document the basic usage of
`gen-manifests-diff`.

Signed-off-by: kksat <22549266+kksat@users.noreply.github.com>

install project dependencies
use latest fedora as base
Signed-off-by: kksat <22549266+kksat@users.noreply.github.com>

Signed-off-by: kksat <22549266+kksat@users.noreply.github.com>

The new `common.Must()` helper can be used in e.g. tests that
want a short way of writing `NewFoo() (Foo, error)` or
`LoadBar() (Bar, error)`, e.g.
```go
map[string]rhel.PackageSetFunc{
    rhel.OSPkgsKey: func(t *rhel.ImageType) rpmmd.PackageSet {
	return common.Must(packagesets.Load(t, "bare-metal", nil))
    },
}
```

This commit makes the yaml loader deal with different layout
for the yaml directories and also deals with the fact that
go:embed does not support symlinks.

This is (hopefully) a temporary measure, I expect that eventually
there will be a `distros.yaml` or similar file that describes
the various distros supported and some details (like their data
dir).

This replaces the rhel10 package sets with yaml files. Note that
it uses `common.Must` as a stopgap for now to bridge between
the packagesets.Load() helper and the rhel PackageSetFunc. This
will just go away once all rhel versions are converted.

This mirrors the Ubuntu images in the MS store. The WSL datasource looks
for userdata on the Windows host.