fix(guest-image): stage /sbin/modprobe so in-guest egress arm autoloads netfilter (RIG-3028) - #759
Merged
mattwilkinsonn merged 2 commits intoAug 31, 2026
Conversation
|
Compass engineering docs preview: https://compass-runner-rig-3028-gues.compass-eng-docs.pages.dev Deployed from |
rigel-mintaka
added a commit
that referenced
this pull request
Aug 30, 2026
…t -f (RIG-3028) Review-round low findings on PR #759, all comment/precision (no behavior change): - boot_microvm_test.go: soften the armRuleset doc-comment — it proves a representative slice of the netfilter autoload chain (nfnetlink, nf_tables, nf_conntrack, nft_ct), not "exactly" every module the base ruleset needs (the base ruleset's interval sets + DNS rules pull further modules the same way via request_module). - boot_microvm_test.go: note the 30s Provision context is a deliberately generous CLIENT bound; the real ceiling is the server's 120s armTimeout. - guest-image/default.nix: reword the modprobe symlink comment to attribute closure inclusion to the ${pkgs.kmod} reference, not the ln flag; drop the inert -f (no prior modprobe name exists to overwrite) to match the sibling `ln -s` for /sbin/init and /lib/modules. Re-verified GREEN on real KVM after the change (rootfs still packs /sbin/modprobe -> kmod; TestInGuestEgressArmAutoloadsNetfilter passes). Refs RIG-3028 Co-authored-by: Matt Wilkinson <matt@rigel.build>
…ds netfilter (RIG-3028) The V3 in-guest egress arm (W1, #754) runs `nft` as guest root, whose first NETLINK_NETFILTER socket open requires the kernel to autoload nfnetlink / nf_tables on demand. The guest has no udev / systemd-modules-load (guestd is PID 1, record §(d)), so on-demand loading rides entirely on the kernel's request_module() usermode-helper path — which execs the binary named by /proc/sys/kernel/modprobe (default /sbin/modprobe, CONFIG_MODPROBE_PATH unset in the pinned kernel). Nothing staged that binary, so request_module was a silent no-op: the arm failed `mnl.c:66: Unable to initialize Netlink socket: Protocol not supported` (EPROTONOSUPPORT), and under §(e) always-arm this reddened every microVM Start. The shipped /lib/modules tree was necessary but not sufficient — the false OQ-3 autoload assumption CI surfaced on the W2 microvm leg. Fix: symlink /sbin/modprobe to kmod's modprobe in the rootfs (kmod, not busybox — modules are .ko.xz and CONFIG_MODULE_DECOMPRESS is unset, so the helper must decompress in userspace, the same reason the initrd already uses kmod). With the helper staged, any module the guest asks for autoloads on demand from the shipped tree via its depmod alias/dep metadata. This is the general request_module mechanism, not a fixed preload, so it also covers egress rulesets beyond the base one: a future user-defined rule pulling a new nft expression module autoloads with no guest-image change. Adds a KVM-gated red-green guard, TestInGuestEgressArmAutoloadsNetfilter: it boots a real guest and Provisions a non-empty nft_script (table + chain + `ct state` rule under `set -eu`), so Provision succeeding proves the whole chain (nfnetlink, nf_tables, nf_conntrack, nft_ct) autoloaded. Verified red on a post-W1/pre-fix rootfs (reproduces `mnl.c:66 EPROTONOSUPPORT`) and green with the fix. Refs RIG-3028 Co-authored-by: Matt Wilkinson <matt@rigel.build>
…t -f (RIG-3028) Review-round low findings on PR #759, all comment/precision (no behavior change): - boot_microvm_test.go: soften the armRuleset doc-comment — it proves a representative slice of the netfilter autoload chain (nfnetlink, nf_tables, nf_conntrack, nft_ct), not "exactly" every module the base ruleset needs (the base ruleset's interval sets + DNS rules pull further modules the same way via request_module). - boot_microvm_test.go: note the 30s Provision context is a deliberately generous CLIENT bound; the real ceiling is the server's 120s armTimeout. - guest-image/default.nix: reword the modprobe symlink comment to attribute closure inclusion to the ${pkgs.kmod} reference, not the ln flag; drop the inert -f (no prior modprobe name exists to overwrite) to match the sibling `ln -s` for /sbin/init and /lib/modules. Re-verified GREEN on real KVM after the change (rootfs still packs /sbin/modprobe -> kmod; TestInGuestEgressArmAutoloadsNetfilter passes). Refs RIG-3028 Co-authored-by: Matt Wilkinson <matt@rigel.build>
rigel-mintaka
force-pushed
the
compass-runner/rig-3028-guest-modprobe-autoload
branch
from
August 31, 2026 01:50
1c05ffa to
0e4b215
Compare
mattwilkinsonn
approved these changes
Aug 31, 2026
mattwilkinsonn
deleted the
compass-runner/rig-3028-guest-modprobe-autoload
branch
August 31, 2026 03:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The V3 in-guest egress arm (W1, #754) runs
nftas guest root, whose firstNETLINK_NETFILTER socket open requires the kernel to autoload nfnetlink /
nf_tables on demand. The guest has no udev / systemd-modules-load (guestd is
PID 1, record §(d)), so on-demand loading rides entirely on the kernel's
request_module() usermode-helper path — which execs the binary named by
/proc/sys/kernel/modprobe (default /sbin/modprobe, CONFIG_MODPROBE_PATH unset in
the pinned kernel). Nothing staged that binary, so request_module was a silent
no-op: the arm failed
mnl.c:66: Unable to initialize Netlink socket: Protocol not supported(EPROTONOSUPPORT), and under §(e) always-arm this reddened everymicroVM Start. The shipped /lib/modules tree was necessary but not sufficient —
the false OQ-3 autoload assumption CI surfaced on the W2 microvm leg.
Fix: symlink /sbin/modprobe to kmod's modprobe in the rootfs (kmod, not busybox
— modules are .ko.xz and CONFIG_MODULE_DECOMPRESS is unset, so the helper must
decompress in userspace, the same reason the initrd already uses kmod). With the
helper staged, any module the guest asks for autoloads on demand from the shipped
tree via its depmod alias/dep metadata. This is the general request_module
mechanism, not a fixed preload, so it also covers egress rulesets beyond the base
one: a future user-defined rule pulling a new nft expression module autoloads
with no guest-image change.
Adds a KVM-gated red-green guard, TestInGuestEgressArmAutoloadsNetfilter: it
boots a real guest and Provisions a non-empty nft_script (table + chain +
ct staterule underset -eu), so Provision succeeding proves the whole chain(nfnetlink, nf_tables, nf_conntrack, nft_ct) autoloaded. Verified red on a
post-W1/pre-fix rootfs (reproduces
mnl.c:66 EPROTONOSUPPORT) and green with thefix.
Refs RIG-3028
Co-authored-by: Matt Wilkinson matt@rigel.build