fix(email_scan): tolerate attributes on screener's errorlist markup - #617
Conversation
screener.py reported "Unexpected response body structure" for every
address. The availability check matched the error list by exact tag text:
'<ul class="errorlist"><li>This field is required.</li>'
screener.in now renders that list with an id, as Django emits on field
errors:
<ul class="errorlist" id="id_password_error"><li>This field is required.</li></ul>
`<ul class="errorlist"` is no longer immediately followed by `>`, so the
substring test failed and every scan fell through to the error branch.
Match the tag by its class and the message it wraps instead, so the check
also survives further attribute additions or reordering. The "already
exists" (taken) branch matches plain text and is unchanged.
Verified live against screener.in with an unregistered address:
before, "Error (Unexpected response body structure)"; after, "Not
Registered". ruff, mypy and pytest all clean.
Fixes kaifcodec#611
|
@ARAVIND281 Can you test it again? |
|
Re-tested just now, and it passes here — so let's find where our setups differ rather than me just saying "works for me". A/B on the same machine, same network, same address, minutes apart: What the site returns right now, replaying the module's own two-step flow: So the markup is still the Most likely explanation: the error message you're seeing is exactly what gh pr checkout 617
python -c "import user_scanner, os; print(os.path.dirname(user_scanner.__file__))" # should be your worktree, not site-packages
python -m user_scanner -e <some-unregistered-address> -m screenerIf it still fails after that, then we differ somewhere I can't see and I'd like the detail rather than guessing. The useful thing would be the raw response — whether One thing I did not test: the |
kaifcodec
left a comment
There was a problem hiding this comment.
@ARAVIND281 Most likely the issue was on my end. I likely had forgot to switch the branch.
LGTM. Merging it now.
Thank you for the PR and feel free to contribute further to the project.
Fixes #611
email_scan/other/screener.pyreturnedUnexpected response body structurefor every address, because the availability check matched the Django error list by exact tag text:screener.in now renders that list with an id:
<ul class="errorlist"is no longer immediately followed by>, so the substring test fails and every scan falls through to the error branch.Fix
Match the tag by its class and the message it wraps, rather than verbatim:
This is slightly broader than the regex suggested in the issue: it also tolerates attribute reordering (
<ul id="..." class="errorlist">), extra classes (class="errorlist nonfield"), and pretty-printed whitespace between the<ul>and its<li>— so the next markup change doesn't reopen this.It stays narrow in the ways that matter: the message text is still required, and the
errorlistclass is matched with\bboundaries, so a different error (Enter a valid email address.) or a different list (class="helptext") is not read as available.The
takenbranch matches plain text (User account with this Email already exists) and is untouched.Verification
Live against screener.in, with an unregistered address:
Local gates from AGENTS.md, all clean:
Note on the
takenpath: I verified theavailablepath live, but didn't probe a real registered account to exercisetaken, since that means testing against someone's actual email. That branch isn't touched by this diff — happy to confirm it another way if you'd prefer.I did initially write a mocked regression test for the marker, then dropped it after re-reading AGENTS.md ("Do not add unit tests for individual scan modules"). Let me know if a test would be welcome here as an exception, since this is a detection-regression rather than a new module — I have it ready.