feat(guestd): arm nft_script as guest root before the exec gate (V3 W1) by rigel-mintaka · Pull Request #754 · RigelBuild/compass

rigel-mintaka · 2026-08-30T20:55:40Z

This PR is part of a stack containing 2 PRs:

main
"feat(guestd): arm nft_script as guest root before the exec gate (V3 W1)" (this PR)
feat(runtime): deliver egress policy to the guest; skip host arm on self-arming backends (V3 W2) #755

Replace the V2b CodeUnimplemented branch in supervisor.Provision with the
in-guest egress arm from the frozen microVM Runner V3 record (§(d)): a non-empty
nft_script runs as guestd's own root, under s.mu, before the state
transition. A failed arm returns CodeInternal and leaves the gate closed at
stateReady, so requireProvisioned keeps refusing every exec and the host
tears the VM down — the fail-closed contract. An empty script skips the arm (the
§(e) hermetic test seam); the host production path never sends empty.

armFunc func(ctx, script) error seam on the supervisor (beside
newCredential), production runNftScript spawning /bin/sh -c <script> with
no syscall.Credential — a spawn path deliberately separate from exec
children, bounded by armTimeout = 120s (OQ-5) and the RPC ctx, carrying the
exit status + bounded output tail on failure (OQ-6 CodeInternal).
Guest image links /bin/sh (bashInteractive, zero added closure): under
always-arm every microVM Start spawns /bin/sh -c, so a missing /bin/sh is
a total-backend outage, not egress-only.
Proto/handler doc-comments describe the as-built arm (§(f)); no wire change.
Tests rewrite the V2b non-empty→CodeUnimplemented assertion into the arm
contract: exact-bytes arm + stateProvisioned; failed arm ⇒ CodeInternal,
gate stays closed, retry allowed; empty script skips the arm;
already-provisioned refusal holds after a successful arm (no wire re-arm); a
real-/bin/sh exit 7 row proves the failure path (host /bin/sh only; the
guest link is proven by the W3/KVM suite).

Refs RIG-3018

Co-authored-by: Matt Wilkinson matt@rigel.build

Replace the V2b `CodeUnimplemented` branch in `supervisor.Provision` with the in-guest egress arm from the frozen microVM Runner V3 record (§(d)): a non-empty `nft_script` runs as guestd's own root, under `s.mu`, before the state transition. A failed arm returns `CodeInternal` and leaves the gate closed at `stateReady`, so `requireProvisioned` keeps refusing every exec and the host tears the VM down — the fail-closed contract. An empty script skips the arm (the §(e) hermetic test seam); the host production path never sends empty. - `armFunc func(ctx, script) error` seam on the supervisor (beside `newCredential`), production `runNftScript` spawning `/bin/sh -c <script>` with no `syscall.Credential` — a spawn path deliberately separate from exec children, bounded by `armTimeout = 120s` (OQ-5) and the RPC ctx, carrying the exit status + bounded output tail on failure (OQ-6 `CodeInternal`). - Guest image links `/bin/sh` (`bashInteractive`, zero added closure): under always-arm every microVM Start spawns `/bin/sh -c`, so a missing `/bin/sh` is a total-backend outage, not egress-only. - Proto/handler doc-comments describe the as-built arm (§(f)); no wire change. - Tests rewrite the V2b non-empty→`CodeUnimplemented` assertion into the arm contract: exact-bytes arm + `stateProvisioned`; failed arm ⇒ `CodeInternal`, gate stays closed, retry allowed; empty script skips the arm; already-provisioned refusal holds after a successful arm (no wire re-arm); a real-`/bin/sh` `exit 7` row proves the failure path (host `/bin/sh` only; the guest link is proven by the W3/KVM suite). Refs RIG-3018 Co-authored-by: Matt Wilkinson <matt@rigel.build>

linear-code · 2026-08-30T20:55:43Z

RIG-3018

github-actions · 2026-08-30T20:57:14Z

Compass engineering docs preview: https://compass-runner-rig-3018-v3-w.compass-eng-docs.pages.dev

Deployed from compass-runner/rig-3018-v3-w1-guestd-arm at 9f48e66.

The V3 egress arm spawns `/bin/sh -c <script>` as a spawn path deliberately separate from exec children (§(d): no credential, no exec table), so it never inherits the PATH floor that `mergeEnv` applies to every exec child. guestd runs as PID 1 with no PATH, so the arm child's shell saw an empty PATH and the script's bare `nft`/`getent`/`awk` (all linked under `/bin`, guest-image/default.nix:321-323) failed `command not found` — failing the arm, failing Provision, and under always-arm (§(e)) failing EVERY microVM Start: the total-backend outage the record flags for this hazard class. Set `cmd.Env = ["PATH=" + defaultGuestPATH]` on the arm spawn so bare tools resolve against the guest rootfs userland, mirroring the exec path's floor. The host `/bin/sh` in hermetic tests always inherited a populated PATH, so this only surfaced when a real guest (PID 1, empty env) ran a real non-empty script — i.e. W1's arm and W2's always-arm script together, exactly the stacked KVM contract run (contract_microvm_test.go). Adds a hermetic regression asserting the arm child's PATH equals defaultGuestPATH (echo/redirection are shell builtins, so the captured value reflects only the env the arm sets); it fails red against the pre-fix inherited-env spawn. Refs RIG-3018 Co-authored-by: Matt Wilkinson <matt@rigel.build>

…ds netfilter (RIG-3028) The V3 in-guest egress arm (W1, #754) runs `nft` as guest root, whose first NETLINK_NETFILTER socket open requires the kernel to autoload nfnetlink / nf_tables on demand. The guest has no udev / systemd-modules-load (guestd is PID 1, record §(d)), so on-demand loading rides entirely on the kernel's request_module() usermode-helper path — which execs the binary named by /proc/sys/kernel/modprobe (default /sbin/modprobe, CONFIG_MODPROBE_PATH unset in the pinned kernel). Nothing staged that binary, so request_module was a silent no-op: the arm failed `mnl.c:66: Unable to initialize Netlink socket: Protocol not supported` (EPROTONOSUPPORT), and under §(e) always-arm this reddened every microVM Start. The shipped /lib/modules tree was necessary but not sufficient — the false OQ-3 autoload assumption CI surfaced on the W2 microvm leg. Fix: symlink /sbin/modprobe to kmod's modprobe in the rootfs (kmod, not busybox — modules are .ko.xz and CONFIG_MODULE_DECOMPRESS is unset, so the helper must decompress in userspace, the same reason the initrd already uses kmod). With the helper staged, any module the guest asks for autoloads on demand from the shipped tree via its depmod alias/dep metadata. This is the general request_module mechanism, not a fixed preload, so it also covers egress rulesets beyond the base one: a future user-defined rule pulling a new nft expression module autoloads with no guest-image change. Adds a KVM-gated red-green guard, TestInGuestEgressArmAutoloadsNetfilter: it boots a real guest and Provisions a non-empty nft_script (table + chain + `ct state` rule under `set -eu`), so Provision succeeding proves the whole chain (nfnetlink, nf_tables, nf_conntrack, nft_ct) autoloaded. Verified red on a post-W1/pre-fix rootfs (reproduces `mnl.c:66 EPROTONOSUPPORT`) and green with the fix. Refs RIG-3028 Co-authored-by: Matt Wilkinson <matt@rigel.build>

…ds netfilter (RIG-3028) (#759) * fix(guest-image): stage /sbin/modprobe so in-guest egress arm autoloads netfilter (RIG-3028) The V3 in-guest egress arm (W1, #754) runs `nft` as guest root, whose first NETLINK_NETFILTER socket open requires the kernel to autoload nfnetlink / nf_tables on demand. The guest has no udev / systemd-modules-load (guestd is PID 1, record §(d)), so on-demand loading rides entirely on the kernel's request_module() usermode-helper path — which execs the binary named by /proc/sys/kernel/modprobe (default /sbin/modprobe, CONFIG_MODPROBE_PATH unset in the pinned kernel). Nothing staged that binary, so request_module was a silent no-op: the arm failed `mnl.c:66: Unable to initialize Netlink socket: Protocol not supported` (EPROTONOSUPPORT), and under §(e) always-arm this reddened every microVM Start. The shipped /lib/modules tree was necessary but not sufficient — the false OQ-3 autoload assumption CI surfaced on the W2 microvm leg. Fix: symlink /sbin/modprobe to kmod's modprobe in the rootfs (kmod, not busybox — modules are .ko.xz and CONFIG_MODULE_DECOMPRESS is unset, so the helper must decompress in userspace, the same reason the initrd already uses kmod). With the helper staged, any module the guest asks for autoloads on demand from the shipped tree via its depmod alias/dep metadata. This is the general request_module mechanism, not a fixed preload, so it also covers egress rulesets beyond the base one: a future user-defined rule pulling a new nft expression module autoloads with no guest-image change. Adds a KVM-gated red-green guard, TestInGuestEgressArmAutoloadsNetfilter: it boots a real guest and Provisions a non-empty nft_script (table + chain + `ct state` rule under `set -eu`), so Provision succeeding proves the whole chain (nfnetlink, nf_tables, nf_conntrack, nft_ct) autoloaded. Verified red on a post-W1/pre-fix rootfs (reproduces `mnl.c:66 EPROTONOSUPPORT`) and green with the fix. Refs RIG-3028 Co-authored-by: Matt Wilkinson <matt@rigel.build> * fix(guest-image): address review nits — comment precision + drop inert -f (RIG-3028) Review-round low findings on PR #759, all comment/precision (no behavior change): - boot_microvm_test.go: soften the armRuleset doc-comment — it proves a representative slice of the netfilter autoload chain (nfnetlink, nf_tables, nf_conntrack, nft_ct), not "exactly" every module the base ruleset needs (the base ruleset's interval sets + DNS rules pull further modules the same way via request_module). - boot_microvm_test.go: note the 30s Provision context is a deliberately generous CLIENT bound; the real ceiling is the server's 120s armTimeout. - guest-image/default.nix: reword the modprobe symlink comment to attribute closure inclusion to the ${pkgs.kmod} reference, not the ln flag; drop the inert -f (no prior modprobe name exists to overwrite) to match the sibling `ln -s` for /sbin/init and /lib/modules. Re-verified GREEN on real KVM after the change (rootfs still packs /sbin/modprobe -> kmod; TestInGuestEgressArmAutoloadsNetfilter passes). Refs RIG-3028 Co-authored-by: Matt Wilkinson <matt@rigel.build> --------- Co-authored-by: Matt Wilkinson <matt@rigel.build>

rigel-mintaka requested a review from mattwilkinsonn as a code owner August 30, 2026 20:55

rigel-mintaka mentioned this pull request Aug 30, 2026

feat(runtime): deliver egress policy to the guest; skip host arm on self-arming backends (V3 W2) #755

Open

mattwilkinsonn approved these changes Aug 30, 2026

View reviewed changes

mattwilkinsonn merged commit 43b98d0 into main Aug 30, 2026
25 of 27 checks passed

mattwilkinsonn deleted the compass-runner/rig-3018-v3-w1-guestd-arm branch August 30, 2026 22:26

rigel-mintaka mentioned this pull request Aug 30, 2026

fix(guest-image): stage /sbin/modprobe so in-guest egress arm autoloads netfilter (RIG-3028) #759

Merged

Navigation Menu

Uh oh!

feat(guestd): arm nft_script as guest root before the exec gate (V3 W1) - #754

feat(guestd): arm nft_script as guest root before the exec gate (V3 W1)#754
mattwilkinsonn merged 2 commits into
mainfrom
compass-runner/rig-3018-v3-w1-guestd-arm

rigel-mintaka commented Aug 30, 2026 •

edited

Loading

Uh oh!

linear-code Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

rigel-mintaka commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear-code Bot commented Aug 30, 2026

Uh oh!

github-actions Bot commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rigel-mintaka commented Aug 30, 2026 •

edited

Loading

github-actions Bot commented Aug 30, 2026 •

edited

Loading