Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurability of cache mode for disks with qemu #67

Closed
olifre opened this issue Oct 25, 2023 · 6 comments
Closed

Configurability of cache mode for disks with qemu #67

olifre opened this issue Oct 25, 2023 · 6 comments

Comments

@olifre
Copy link

olifre commented Oct 25, 2023

For most CI use cases, using unsafe instead of writeback is very reasonable for a significant performance gain. It seems reasonable whenever the file systems on the actual disk images do not need to stay intact in case of unexpected hardware (or kernel) failure, which is a reasonable assumption for CI (if it fails, it fails, no salvaging from the broken disk images will be done usually).

Behaviour is effectively similar to ignoring fsync() system calls, which e.g. eatmydata does which is commonly used on Debian systems e.g. for package builds to gain performance.

So my proposal would be to allow to configure the cache mode for qemu VMs, currently, writeback is hardcoded (which of course is a reasonable default, since it works well for any usecase):
https://github.com/cross-platform-actions/action/blob/de0e1f18a909bf9562ca13842e70a71b8b0c8938/src/qemu_vm.ts#L60C1-L64

@jacob-carlborg
Copy link
Contributor

jacob-carlborg commented Oct 26, 2023

Seems reasonable. Is there a reason to use writeback for this action at all, or can we just hardcode unsafe instead? Is there a risk to cause CI failures what would otherwise not occur?

@olifre
Copy link
Author

olifre commented Oct 27, 2023

Pondering about this a bit, I don't really see any reason for this action to use writeback. If the kernel, the hypervisor or the underlying hardware have a problem, CI fails anyways, and unsafe is safe as long as none of these break apart 😉 .

@jacob-carlborg
Copy link
Contributor

Then I suggest we just change the hardcoded value to unsafe. This will make the API and the implementation of the action simpler than if it was configurable. Does that sound good?

@olifre
Copy link
Author

olifre commented Oct 27, 2023

Yes, that sounds good to me 👍.

@olifre
Copy link
Author

olifre commented Oct 27, 2023

One more thought: Of course, if GitHub already uses unsafe (or whatever is equivalent in the underlying virtualization they use), the gains might be negligible. But it is still better IMHO to choose unsafe here, since this will work independent of GitHub's infrastructure and also for self-hosted runners (in case someone uses this action with self-hosted runners).

@manxorist
Copy link

I agree, I think the default can be unsafe, and writeback can be provided as an option. The only use case I can imagine that could conflict with unsafe caching would be kernel (module/driver) development that expects kernel crashes of the OS inside the VM, and wants to analyze collected data in a following step. However, I am not sure if cross-platform-actions would be even suitable for that use case as the OS image (and thus kernel) is already provided.

korli added a commit to korli/action that referenced this issue Mar 15, 2024
Added
- Added support for using the action in multiple steps in the same job ([cross-platform-actions#26](cross-platform-actions#26)).
    All the inputs need to be the same for all steps, except for the following
    inputs: `sync_files`, `shutdown_vm` and `run`.

- Added support for specifying that the VM should not shutdown after the action
    has run. This adds a new input parameter: `shutdown_vm`. When set to `false`,
    this will hopefully mitigate very frequent freezing of VM during teardown ([cross-platform-actions#61](cross-platform-actions#61), [cross-platform-actions#72](cross-platform-actions#72)).

Changed
- Always terminate VM instead of shutting down. This is more efficient and this
    will hopefully mitigate very frequent freezing of VM during teardown
    ([cross-platform-actions#61](cross-platform-actions#61),
    [cross-platform-actions#72](cross-platform-actions#72)).

- Use `unsafe` as the cache mode for QEMU disks. This should improve performance ([cross-platform-actions#67](cross-platform-actions#67)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants