`wit_paths` reordered by `BTreeSet` and do not appear to keep more than the immediately previous `.wit` in scope when running `componentize-go bindings`

[seanwatters]

I'm currently working on a project where I've successfully managed to wire up a Rust equivalent to what I'm now trying to set up for a Golang guest component; wit_bindgen::generate! in Rust works like I'd expect it to, but both the command-line flags variant and the componentize-go.toml variants fail for Go.

Working Rust

In the Rust version I have the wit_bindgen setup:

// lib.rs
wit_bindgen::generate!({
    path: [".ordinary/post/host.wit", ".ordinary/post/harness.wit", ".ordinary/post/function.wit", ".ordinary/post/bridge.wit"],
    world: "ordinary:function/function",
    generate_all,
});

ordinary:host/host being the top-level that all subsequent worlds depend on, ordinary:harness/harness depending only on host but imports ordinary:function/function which imports host and ordinary:bridge/bridge which also imports host. This version works and I'm happy with it (if it helps the wac plug is below to illustrate further)

# wac

# build the FFI harness function call bindings to a WASI component
cargo build --target wasm32-wasip2 --release --manifest-path .ordinary/post/harness/Cargo.toml
# build the FFI application bridge call bindings to a WASI component
cargo build --target wasm32-wasip2 --release --manifest-path .ordinary/post/bridge/Cargo.toml

# build the userland function WASI component
cargo build --target wasm32-wasip2 --release

# plug the bridge WASI component into the userland function WASI component
if wac plug target/wasm32-wasip2/release/function.wasm \
  --plug .ordinary/post/bridge/target/wasm32-wasip2/release/bridge.wasm \
  -o .ordinary/post/intermediate.wasm ; then
    echo no bridged components were used
else
    cp target/wasm32-wasip2/release/function.wasm .ordinary/post/intermediate.wasm
fi

# plug the intermediate component created by function/bride into the harness WASI component
wac plug .ordinary/post/harness/target/wasm32-wasip2/release/harness.wasm \
  --plug .ordinary/post/intermediate.wasm \
  -o .ordinary/post/function.wasm

Broken Go

Now, with the following componentize-go.toml I get an error.

# componentize-go.toml

worlds = ["ordinary:function/function"]
wit_paths = [".ordinary/test/host.wit", ".ordinary/test/harness.wit", ".ordinary/test/function.with", ".ordinary/test/bridge.wit"]

# error
Error: package 'ordinary:host@1.0.0' not found. known packages:
    ordinary:bridge@1.0.0

When I prefix them with a-, b-, etc. to force sort order I get the error that indicates to me that only the immediately previous world is kept in context.

# componentize-go.toml

worlds = ["ordinary:function/function"]
wit_paths = [".ordinary/test/a-host.wit", ".ordinary/test/b-harness.wit", ".ordinary/test/c-function.with", ".ordinary/test/d-bridge.wit"]

# error
Error: package 'ordinary:host@1.0.0' not found. known packages:
    ordinary:harness@1.0.0

What I Found

I was able to get as far as finding that the sort-order mattered here but am not familiar enough with the wit_parser library, yet, to say what/why on the only referencing one world/wit file at a time. The Rust macro does care about/respect order, I don't know why the Go parser wouldn't but I am curious if one does and the other shouldn't why that would be.

EDIT:

If it helps to have a more complete working Rust example, there is one here. cargo test -p ordinary-function should generate the *.wit files in a gitignored .ordinary.

[seanwatters]

After spending a couple hours working against the same issue with componentize-js it seems like there is an implicit (though I'm not sure where it's documented) directory structure that does work if you adhere to it (and works in the Rust version as well) that is:

|- wit
    |- deps
        |- your-package-snake-cased-1.2.3
            |- package.wit
    |- function.wit 
|- componentize-go.toml

# componentize-go.toml
worlds = ["ordinary:function/function"]
wit_paths = ["wit"]