Improve negative narrowing for literal expressions in containers - #21914
Open
Om-singhaI wants to merge 1 commit into
Open
Improve negative narrowing for literal expressions in containers#21914Om-singhaI wants to merge 1 commit into
Om-singhaI wants to merge 1 commit into
Conversation
`x in ('a', 'b')` left the else branch at the full declared type, while
the annotated form `x in t` for `t: tuple[Literal['a'], Literal['b']]`
already narrowed it. The gate for negative narrowing asks
`is_singleton_equality_type`, which only accepts a `LiteralType`, but a
literal written inline arrives as an `Instance` carrying a
`last_known_value`. Coerce before asking.
Only the gate coerces. The type handed to
`narrow_type_by_identity_equality` is untouched, so the positive branch
keeps matching `==`.
Contributor
|
Diff from mypy_primer, showing the effect of this PR on open source code: discord.py (https://github.com/Rapptz/discord.py)
- discord/components.py:1750: error: Missing return statement [return]
pydantic (https://github.com/pydantic/pydantic)
- pydantic/v1/types.py:704: error: Unsupported operand types for >= ("Literal['n']" and "int") [operator]
- pydantic/v1/types.py:704: error: Unsupported operand types for >= ("Literal['N']" and "int") [operator]
- pydantic/v1/types.py:704: error: Unsupported operand types for >= ("Literal['F']" and "int") [operator]
- pydantic/v1/types.py:704: note: Left operand is of type "Literal['n', 'N', 'F'] | int"
- pydantic/v1/types.py:706: error: Unsupported operand types for + ("int" and "Literal['n']") [operator]
- pydantic/v1/types.py:706: error: Unsupported operand types for + ("int" and "Literal['N']") [operator]
- pydantic/v1/types.py:706: error: Unsupported operand types for + ("int" and "Literal['F']") [operator]
- pydantic/v1/types.py:706: note: Right operand is of type "Literal['n', 'N', 'F'] | int"
- pydantic/v1/types.py:714: error: Argument 1 to "abs" has incompatible type "Literal['n', 'N', 'F'] | int"; expected "SupportsAbs[int]" [arg-type]
- pydantic/v1/types.py:715: error: Argument 1 to "abs" has incompatible type "Literal['n', 'N', 'F'] | int"; expected "SupportsAbs[int]" [arg-type]
- pydantic/v1/types.py:718: error: Argument 1 to "abs" has incompatible type "Literal['n', 'N', 'F'] | int"; expected "SupportsAbs[int]" [arg-type]
pytest-autoprofile (https://gitlab.com/TTsangSC/pytest-autoprofile)
+ tests/test_subprocess.py:382: error: Redundant cast to "tuple[Literal[0, 1, 2], Literal[0, 1, 2]]" [redundant-cast]
|
Author
|
Primer looks good. The pydantic and discord.py diffs are false positives going away: both narrow a The one new diagnostic is correct. I expected some new unreachable errors and there are none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A membership test against a container written out as a literal expression does not
narrow the negative branch:
With this change that reveal is
Literal['a'].The same container behind an annotation narrows fine, and so does the equivalent
!=chain, so today the result depends on how the container is spelled. Tuple,list, set and dict literals all behave this way. The practical cost is that
assert_neveroverLiteralalternatives is rejected when the branches arewritten with
inand accepted when they are written with==, and that thecommon
VALID: Final = ("a", "b")idiom followed byif x not in VALIDleavesxat its declared type. That last one is the tuple spelling. AFinallist isstill not narrowed, since it infers as
list[str]and the item type is gone bythe time the container is inspected.
This is a recorded gap rather than deliberate conservatism:
narrow_tuple_expressionin check-narrowing.test carries a
# TODO: this should match narrow_tuple_exactmarker on exactly this behaviour, added in #21456, and #21461 extended the same gap
to list, set and dict literals. I could not find an open issue for it.
Cause
The gate that decides whether the else map can be kept asks
is_singleton_equality_type, which accepts aLiteralType. A literal writteninline never reaches that point as a
LiteralType: it arrives as anInstanceofbuiltins.strcarryinglast_known_value=Literal['r']. The coercion just aboveonly fires for what
is_literal_type_likerecognises, which isLiteralType,unions and type variables, plus a separate clause for enums. That is why enum
members and annotated containers already narrow while plain
strandintliterals do not. The else map is computed correctly and then dropped on the floor.
Fix
Coerce the container item before asking whether it denotes a single value.
Only the gate coerces. The type passed to
narrow_type_by_identity_equalityisleft as it was, so the positive branch is unchanged:
x in ["x"]forx: str | intstill revealsbuiltins.str, matchingx == "x"(
testConsistentNarrowingEqAndIn, #17864). Coercing the item itself is theshorter patch, but I tried it and it turns that reveal into
Literal['x']andadds a spurious assignment error, so the coercion has to stay local to the gate.
The custom
__eq__exemption is carried over from the block above, sotestNarrowCustomEqEnumInLiteralContainer(#21703) still declines to narrow.Soundness
x not in (a, b)should land in the same place asx != a and x != b. I checkedthat by pairing every case against its
!=control:boolagainst anintliteral,
intagainstfloat, a class with a custom__eq__inside the narrowedunion, a custom
__eq__enum, astrsubclass in the container,Anyin thecontainer, a plain variable, a container mixing a literal with a variable, and
Nonemixed with string literals. Everynot inresult matches its!=controlexactly. Only five of those twenty reveals move at all against master, and each one
moves from unnarrowed to whatever
!=already produced.Tests
testNarrowLiteralInLiteralContainerand removed the now stale TODO.testNarrowNotInLiteralContainer, coveringintliterals, aFinaltuple constant (which reaches the
TupleTypebranch rather than the tupleexpression one), a container mixing a literal with
None, exhaustiveness, andtwo cases that must stay wide: a container holding a plain variable, and
FalseagainstLiteral[0, 1, 2].mypy/checker.pyand keeping the test data makes both cases failat nine sites. The two cases that must stay wide are identical either way.
optional, python310, typeddict, tuples, inference, flags, statements and
classes test files locally; all pass.
black,ruff checkandcodespellareclean on the changed file, and mypy's self check reports nothing new.
I expect mypy_primer to surface new
unreachablediagnostics, since code after anexhaustive
inchain now genuinely is unreachable.