fix: add timeout and scheme validation when fetching remote topologies - #3767
Open
bunlongheng wants to merge 1 commit into
Open
fix: add timeout and scheme validation when fetching remote topologies#3767bunlongheng wants to merge 1 commit into
bunlongheng wants to merge 1 commit into
Conversation
Owner
|
Nice fixes, thank you. However, being of the "every constant eventually becomes a variable" persuasion, it would be nice to have both parameters (timeout, allowed URL schemas) as variables defined in system settings, which creates a bit of a conundrum as we have to read the topology before we can download the topology. I will add that bit to the code (might take a few days to find the time to do it) and then merge this. Thanks again, Ivan |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
http_fetch_content()innetsim/cli/create.pydownloads a remote topology withrequests.get(url)using no timeout and without validating the URL scheme. This function is reached whenever a topology is supplied as a URL (netlab up <url>/netlab create <url>), and it is also reachable from the built-innetlab apiserver: aPOST /jobsrequest with atopologyUrlfield routes intonetlab create/netlab up, which call this function.Two problems:
No request timeout.
requests.get()defaults to blocking forever. In thenetlab apiserver the fetch runs inside a worker that holds the globalRUN_LOCK(seenetsim/cli/api.py), so a single request pointing at an unresponsive or slow-loris host makes the worker hang indefinitely and permanently wedges the job queue. Basic auth on the API server is optional and off by default, so this is triggerable without credentials.No URL-scheme validation. The URL is passed straight to
requestswith no restriction, so unexpected schemes (for examplefile://when a filesystem transport adapter is mounted) are accepted rather than rejected up front.Fix
http/httpsbefore the request is made.HTTP_FETCH_TIMEOUT(20s) to therequests.get()call so a stuck download fails cleanly instead of blocking forever.The change is confined to
http_fetch_content()and preserves existing behavior for validhttp/httpstopology URLs (including the GitHub?raw=truerewrite).Testing
python3 -c "import ast; ast.parse(open('netsim/cli/create.py').read())"passes.http/httpsURLs follow the unchanged download path; non-http(s) URLs now exit with a clear error; an unresponsive host now fails after the timeout instead of hanging.