test(sandbox): cover the modal and cloudflare bucket mount strategies - #4246
test(sandbox): cover the modal and cloudflare bucket mount strategies#4246abhay-codes07 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 64f7b1f24a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| import pytest | ||
|
|
||
| from agents.extensions.sandbox.modal.mounts import ModalCloudBucketMountStrategy |
There was a problem hiding this comment.
Avoid collecting Modal tests through the real extra
In a dev or packaged-test environment that does not install the optional modal extra, this top-level submodule import still executes agents.extensions.sandbox.modal.__init__, which imports .sandbox and then import modal, so pytest collection fails before these characterization tests can run. The existing Modal tests avoid that by installing a fake modal module before importing the backend; please load this strategy through that helper or otherwise skip/fake the optional dependency.
AGENTS.md reference: AGENTS.md:L78-L86
Useful? React with 👍 / 👎.
|
Thanks for sharing these changes. I don't have the bandwidth to check this type of changes right now, so holding off sending simple test changes just for coverage improvement would be appreciated. |
Completes coverage of the sandbox provider mount strategies, taking modal/mounts.py and cloudflare/mounts.py to 100%. Config building was already tested for both. The gap was the strategy lifecycle and the error branches, which normally need a live provider account to reach. Cloudflare: - activate and deactivate drive mount_bucket and unmount_bucket with the resolved mount path and the built request options. - The snapshot hooks unmount and then remount the same path, so a snapshot cannot capture a live bucket and the mount returns after. - All four lifecycle methods reject a session from another backend, asserted by checking that no bucket call was made at all. - Prefix normalisation across none, empty, bare and wrapped forms, a half-supplied credential pair, and an unsupported mount type. Modal: - supports_native_snapshot_detach is False, which is what routes snapshots through _native_snapshot_requires_tar_fallback rather than trusting a native snapshot taken while a bucket is attached. That link was previously unpinned. - activate and the snapshot hooks are no-ops, because modal attaches buckets at sandbox create time, and activate and deactivate still reject a foreign session. - Secret option validation, and the rule that inline credentials cannot be combined with a named secret, for s3, r2 and gcs. Bucket calls are recorded on a single ordered timeline, so the tests assert an exact call sequence rather than checking two independent lists that would not constrain their relative order. The modal tests stub the optional modal SDK when it is absent, since the package __init__ imports the backend at module scope and would otherwise fail collection in an environment without the extra. No source changes.
64f7b1f to
76a3555
Compare
|
Thanks, the Codex catch is right and is now fixed. Optional
I also tightened the bucket-call assertions. These were checking assert session.events == [f"unmount:{path}", f"mount:{path}"]To be accurate about what that buys: for cloudflare the ordering was already implied by the test calling the two hooks in sequence, so this is not fixing a false pass the way it was on #4240. What it does add is that the exact call count and the absence of stray bucket calls are now pinned, and the foreign-session guard asserts Rebased on
The full suite was run on Windows, where some sandbox symlink and tracing timing tests fail independently of this change. I diffed the failing set against clean |
|
@abhay-codes07 I should have mentioned you in the previous comment, but I don't have the bandwidth to review this type of PRs at this moment. Keeping this one is okay, but please hold off sending similar ones (I won't actively review those). |
|
Understood, thanks for saying it plainly. I will stop sending coverage only test PRs. No rush on this one, pick it up whenever it suits you or leave it. |
Summary
Completes coverage of the sandbox provider mount strategies, following #4240 which did e2b and runloop.
extensions/sandbox/modal/mounts.pyextensions/sandbox/cloudflare/mounts.pyConfig building was already tested for both. The gap was the strategy lifecycle and the error branches, which normally need a live provider account to reach.
Cloudflare
activateanddeactivatedrivemount_bucketandunmount_bucketwith the resolved mount path and the built request options.None, empty, bare and wrapped forms, a half-supplied credential pair, and an unsupported mount type.Modal
supports_native_snapshot_detachisFalse, which is what routes snapshots through_native_snapshot_requires_tar_fallbackinstead of trusting a native snapshot taken while a bucket is attached. That link was previously unpinned, so it is worth having a test on it.activateand the snapshot hooks are no-ops, because Modal attaches buckets at sandbox create time, andactivateanddeactivatestill reject a foreign session.No source changes. Every assertion matches current behaviour, so this is a characterisation of what the code does today.
Test plan
New
tests/extensions/sandbox/test_modal_mounts.py, following the existingtest_blaxel_mounts.pyconvention, plus additions totests/extensions/sandbox/test_cloudflare.py. Cloudflare uses a recording session that capturesmount_bucketandunmount_bucketcalls on a single ordered timeline, so the tests assert an exact call sequence rather than checking two independent lists. Modal's strategy only inspects the session class name, so a bare stand-in is enough; the file stubs the optionalmodalSDK when it is absent, since the package__init__imports the backend at module scope and would otherwise fail collection.make formatmake lintmake mypymain, none in the touched filesmake pyrightuv run pytest tests/extensions/sandbox/make testsThe full suite run was done on Windows, where a set of sandbox symlink and tracing timing tests fail independently of this change. I diffed the failing set against clean
mainin the same environment: no new failures, with 35 more tests passing here.Issue number
None. This is coverage work on the sandbox providers rather than a reported defect.
Checks
.agents/skills/code-change-verification/scripts/run.sh/reviewbefore submitting this PRThe verification script is a bash script that shells out to
make. I ran the underlying steps individually instead, with the results above.