Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for custom image sizes #1826

Merged
merged 2 commits into from Nov 18, 2021

Conversation

bcressey
Copy link
Contributor

@bcressey bcressey commented Nov 15, 2021

Issue number:
Fixes #1821

Description of changes:
Refactor partition layout code into a helper library, and add support for custom OS and data image sizes in the variant specification.

Fix a bug that affected the last partition on each disk, where it was one sector larger than intended. (The corresponding filesystems were already the correct size.)

Testing done:

  • verify that the partition offsets are the same (modulo fix) for the default layout
  • verify that an undersized hash partition is rejected by the new check
  • verify that in-place upgrade works for the default layout, since this touches the GPT priority bits
  • verify that in-place upgrade works for custom layouts with a larger OS image

Confirmed that the "old" and "new" layouts only differ by one sector in the last partition.

Old layout

Sector size (logical): 512 bytes
Disk identifier (GUID): 309D0119-AA57-4863-B5DE-37833FEE0E97
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 4194270
Partitions will be aligned on 2048-sector boundaries
Total free space is 4028 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           10239   4.0 MiB     EF02  BIOS-BOOT
   2           10240           20479   5.0 MiB     EF00  EFI-SYSTEM
   3           20480          102399   40.0 MiB    FFFF  BOTTLEROCKET-BOOT-A
   4          102400         1986559   920.0 MiB   FFFF  BOTTLEROCKET-ROOT-A
   5         1986560         2007039   10.0 MiB    FFFF  BOTTLEROCKET-HASH-A
   6         2007040         2058239   25.0 MiB    FFFF  BOTTLEROCKET-RESERV...
   7         2058240         2068479   5.0 MiB     FFFF  EFI-BACKUP
   8         2068480         2150399   40.0 MiB    FFFF  BOTTLEROCKET-BOOT-B
   9         2150400         4034559   920.0 MiB   FFFF  BOTTLEROCKET-ROOT-B
  10         4034560         4055039   10.0 MiB    FFFF  BOTTLEROCKET-HASH-B
  11         4055040         4106239   25.0 MiB    FFFF  BOTTLEROCKET-RESERV...
  12         4106240         4192256   42.0 MiB    FFFF  BOTTLEROCKET-PRIVATE

New layout

Sector size (logical): 512 bytes
Disk identifier (GUID): 1754185D-33D2-4008-A1DA-FC84B274EFFB
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 4194270
Partitions will be aligned on 2048-sector boundaries
Total free space is 4029 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           10239   4.0 MiB     EF02  BIOS-BOOT
   2           10240           20479   5.0 MiB     EF00  EFI-SYSTEM
   3           20480          102399   40.0 MiB    FFFF  BOTTLEROCKET-BOOT-A
   4          102400         1986559   920.0 MiB   FFFF  BOTTLEROCKET-ROOT-A
   5         1986560         2007039   10.0 MiB    FFFF  BOTTLEROCKET-HASH-A
   6         2007040         2058239   25.0 MiB    FFFF  BOTTLEROCKET-RESERV...
   7         2058240         2068479   5.0 MiB     FFFF  EFI-BACKUP
   8         2068480         2150399   40.0 MiB    FFFF  BOTTLEROCKET-BOOT-B
   9         2150400         4034559   920.0 MiB   FFFF  BOTTLEROCKET-ROOT-B
  10         4034560         4055039   10.0 MiB    FFFF  BOTTLEROCKET-HASH-B
  11         4055040         4106239   25.0 MiB    FFFF  BOTTLEROCKET-RESERV...
  12         4106240         4192255   42.0 MiB    FFFF  BOTTLEROCKET-PRIVATE

Confirmed that an undersized hash partition is rejected if I reduce HASH_SCALE_FACTOR to 2.

  #12 18.32 verity content is larger than partition (4M)
  #12 ERROR: executor failed running [/bin/sh -c /host/tools/rpm2img ...

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

Copy link
Member

@jpculp jpculp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems OK to me pending testing.

tools/rpm2img Outdated Show resolved Hide resolved
Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whew nice work! 🧇

tools/buildsys/src/manifest.rs Show resolved Hide resolved
Copy link
Contributor

@tjkirch tjkirch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it, it's just clever enough to get the flexibility we need without the logic being too hard to read. The math involved, though, is tough and non-obvious - I think it'd help to have an ASCII/draw.io diagram of an example partition layout with an explanation of MB vs. MiB vs. GiB vs. sectors where appropriate. The old sgdisk call at least had the partitions laid out next to each other, which helped a bit, even though it was hiding some secrets.

LGTM if upgrade/downgrade testing comes out OK.

@bcressey bcressey marked this pull request as ready for review November 17, 2021 00:09
@bcressey
Copy link
Contributor Author

Fix or selectively disable shellcheck warnings.

tools/rpm2img Outdated Show resolved Hide resolved
tools/rpm2img Outdated Show resolved Hide resolved
Move all decisions about partition sizes, types, etc into a library
so that it's easier to support layouts for different disk sizes. The
layout is still essentially static, but can be scaled up as needed to
support larger root filesystems.

For the 2 GiB OS image layout, the end result is almost identical,
except for fixing a longstanding bug where the last two partitions on
each disk - PRIVATE and DATA - were (1 MiB + 1 sector) in size, and
not 1 MiB as intended.

Signed-off-by: Ben Cressey <bcressey@amazon.com>
Add a new structure to variant metadata so the OS and data disk sizes
can be overridden. The previous values - 2 GiB for the OS disk, and 1
GIB for the data disk, are promoted to the default values so existing
variant definitions continue working.

References to "disk image" in `rpm2img` have been changed to refer to
"os image" instead, to avoid confusion over which image is meant. The
expected sizes for these images are now required parameters.

Signed-off-by: Ben Cressey <bcressey@amazon.com>
Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛴

@bcressey bcressey merged commit 00ecda4 into bottlerocket-os:develop Nov 18, 2021
@bcressey bcressey deleted the custom-image-size branch November 18, 2021 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Alternate image layout in rpm2img
4 participants