Skip to content

cloud-hypervisor: add platformOEMStrings and --platform merging #336

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

Ramblurr
Copy link
Contributor

The cloud-hypervisor command line interface unfortunately doesn't support
multiple instances of the same arg with a different value, so we have to resort
to these extra module options rather than using extraArgs.

To make matters even worse, the --platform argument (of which there can be
only one), is overloaded with different types of sub-args that also need to be
provided multiple times.

This commit allows the operator to add oem strings (for example to pass systemd
credentials), as well as raw platform options as needed.

Copy link

nix-ci-app bot commented Feb 18, 2025

NixCI is ready to run on this PR.
Maintainer: Comment nix-ci run to run now.

Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Feb 18, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request May 20, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 16, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
@Ramblurr Ramblurr force-pushed the feat/expose-cloud-hypervisor-platform-opts branch from 00c891b to df9df1b Compare July 16, 2025 06:22
@Ramblurr
Copy link
Contributor Author

Ramblurr commented Jul 16, 2025

Hi again @astro, I want to pick this little project back up.. so I'm going to be reviving these PRs today :) Please see my comments to your comments!.

I've just pushed new commits that should address your concerns.

Note

I added the refactor as a new commit, but I expect this to be squashed during the merge with the newest commit's commit message. I wanted to keep the old extraPlatformOpts around in case we changed our minds.

Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 16, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
@Ramblurr Ramblurr changed the title cloud-hypervisor: add platformOEMStrings and extraPlatformOpts. cloud-hypervisor: add platformOEMStrings and --platform merging Jul 16, 2025
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 16, 2025
Previously, cloud-hypervisor always configured vsock with cid=3, causing
problems when runnning multiple VMs from running on the same host due to CID
conflicts. This change respects `microvm.vsock.cid` when set and allows users to
configure vsock via `microvm.cloud-hypervisor.extraArgs`.

The implementation:
- Uses `microvm.vsock.cid` when specified
- Extracts and merges `--vsock` options from extraArgs
- Throws an error if both vsock.cid and --vsock cid=... are provided
- Warns users when vsock is not configured (disabling systemd-notify)

Fixes: microvm-nix#378
Builds on and requires PR microvm-nix#336 (for the extractOptValues function)
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 16, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
@Ramblurr Ramblurr force-pushed the feat/expose-cloud-hypervisor-platform-opts branch 2 times, most recently from 2379b94 to 20d53d9 Compare July 18, 2025 07:24
@Ramblurr
Copy link
Contributor Author

@SuperSandro2000 I've pushed a new commit that rebases onto main to pickup your latest PR, and also addresses your comments. Let me know if there is anything else.

@Ramblurr Ramblurr mentioned this pull request Jul 18, 2025
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 18, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 18, 2025
Previously, cloud-hypervisor always configured vsock with cid=3, causing
problems when runnning multiple VMs from running on the same host due to CID
conflicts. This change respects `microvm.vsock.cid` when set and allows users to
configure vsock via `microvm.cloud-hypervisor.extraArgs`.

The implementation:
- Uses `microvm.vsock.cid` when specified
- Extracts and merges `--vsock` options from extraArgs
- Throws an error if both vsock.cid and --vsock cid=... are provided
- Warns users when vsock is not configured (disabling systemd-notify)

Fixes: microvm-nix#378
Builds on and requires PR microvm-nix#336 (for the extractOptValues function)
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 18, 2025
Previously, cloud-hypervisor always configured vsock with cid=3, causing
problems when runnning multiple VMs from running on the same host due to CID
conflicts. This change respects `microvm.vsock.cid` when set and allows users to
configure vsock via `microvm.cloud-hypervisor.extraArgs`.

The implementation:
- Uses `microvm.vsock.cid` when specified
- Extracts and merges `--vsock` options from extraArgs
- Throws an error if both vsock.cid and --vsock cid=... are provided
- Warns users when vsock is not configured (disabling systemd-notify)

Fixes: microvm-nix#378
Builds on and requires PR microvm-nix#336 (for the extractOptValues function)
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 18, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 18, 2025
Previously, cloud-hypervisor always configured vsock with cid=3, causing
problems when runnning multiple VMs from running on the same host due to CID
conflicts. This change respects `microvm.vsock.cid` when set and allows users to
configure vsock via `microvm.cloud-hypervisor.extraArgs`.

The implementation:
- Uses `microvm.vsock.cid` when specified
- Extracts and merges `--vsock` options from extraArgs
- Throws an error if both vsock.cid and --vsock cid=... are provided
- Warns users when vsock is not configured (disabling systemd-notify)

Fixes: microvm-nix#378
Builds on and requires PR microvm-nix#336 (for the extractOptValues function)
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 18, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
Ramblurr added 2 commits July 18, 2025 15:42
The cloud-hypervisor command line interface unfortunately doesn't support
multiple instances of the same arg with a different value, so we have to resort
to these extra module options rather than using extraArgs.

To make matters even worse, the `--platform` argument (of which there can be
only one), is overloaded with different types of sub-args that also need to be
provided multiple times.

This commit allows the operator to add oem strings (for example to pass systemd
credentials), as well as raw platform options as needed.
Cloud-hypervisor only supports a single --platform argument with comma-separated
options. This commit adds proper handling for platform configuration:

- Add platformOEMStrings option for passing OEM strings in a structured way
- Introduce extractOptValues helper function to extract and remove command-line
  options from argument lists
- Parse any --platform arguments from extraArgs and merge them with configured
  platform options

The extractOptValues function enables proper handling of --platform arguments
in extraArgs while avoiding conflicts with the internally generated --platform
option.
@Ramblurr Ramblurr force-pushed the feat/expose-cloud-hypervisor-platform-opts branch from 20d53d9 to 42eca33 Compare July 18, 2025 13:42
@Ramblurr
Copy link
Contributor Author

fixed the merge conflict

Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 22, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
Ramblurr added a commit to Ramblurr/microvm.nix that referenced this pull request Jul 22, 2025
This commit implements `microvm.credentialFiles` a mechanism for passing
credentials into guest vms from the host.

Currently only support for qemu is implemented as I want to test the waters to
see if you're interested in this feature, Astro.

In addition to qmeu cloud-hypervisor can be supported via smbios. But it depends
on [this feature being added](cloud-hypervisor/cloud-hypervisor#6951 (comment)), and also microvm-nix#336 being merged to microvm.nix

cloud-hypervisor could be supported immediately, but then the secrets would be
visible in the ps output.

A cursory code search shows that the following additional hypervisors could be
supported:

- crosvm: via fw_cfg, or smbios
- alioth: via fw_cfg
- stratovirt: via fw_cfg (maybe smbios)

kvmtool and firecracker both seem like they cannot be supported.

Related:
- microvm-nix#259
- microvm-nix#52
@SuperSandro2000
Copy link
Contributor

Let's try this 🎉 thanks

@SuperSandro2000 SuperSandro2000 merged commit 3b39c12 into microvm-nix:main Jul 22, 2025
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants