Skip to content

Pass most conformance tests for Zuban #2139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
davidhalter wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Pass most conformance tests for Zuban #2139

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5e66e5e
Delete the old descriptions and status for Zuban
davidhalter Dec 20, 2025
deaddd8
Change the flags for Zuban
davidhalter Dec 20, 2025
bdd5006
Make tests a bit easier to work with for Zuban
davidhalter Dec 20, 2025
593d74d
Make it possible to infer more precise types in overloads_evaluation
davidhalter Dec 20, 2025
04c0df5
Methods may be inferred as callables with a 'Self' TypeVar that has a…
davidhalter Dec 20, 2025
8cfbc5a
Allow type checkers to disallow TypeVars in a nested class' __init__
davidhalter Dec 20, 2025
ab19cfd
Run conformance tests
davidhalter Dec 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions conformance/results/mypy/generics_defaults.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ generics_defaults.py:156: error: Expression is of type "type[Foo6[*tuple[Any, ..
generics_defaults.py:156: error: Can only replace ParamSpec with a parameter types list or another ParamSpec, got "str" [misc]
generics_defaults.py:157: error: Expression is of type "type[Foo6[*tuple[Any, ...], Any]]", not "type[Foo6[int, str, [bytes]]]" [assert-type]
generics_defaults.py:157: error: Can only replace ParamSpec with a parameter types list or another ParamSpec, got "str" [misc]
generics_defaults.py:171: error: Expression is of type "Callable[[Self], Self]", not "Callable[[Foo7[int]], Foo7[int]]" [assert-type]
"""
conformance_automated = "Fail"
errors_diff = """
Expand All @@ -36,5 +35,4 @@ Line 79: Unexpected errors ['generics_defaults.py:79: error: Expression is of ty
Line 94: Unexpected errors ['generics_defaults.py:94: error: Expression is of type "type[Class_TypeVarTuple[*DefaultTs]]", not "type[Class_TypeVarTuple[str, int]]" [assert-type]']
Line 156: Unexpected errors ['generics_defaults.py:156: error: Expression is of type "type[Foo6[*tuple[Any, ...], Any]]", not "type[Foo6[int, str, [float, bool]]]" [assert-type]', 'generics_defaults.py:156: error: Can only replace ParamSpec with a parameter types list or another ParamSpec, got "str" [misc]']
Line 157: Unexpected errors ['generics_defaults.py:157: error: Expression is of type "type[Foo6[*tuple[Any, ...], Any]]", not "type[Foo6[int, str, [bytes]]]" [assert-type]', 'generics_defaults.py:157: error: Can only replace ParamSpec with a parameter types list or another ParamSpec, got "str" [misc]']
Line 171: Unexpected errors ['generics_defaults.py:171: error: Expression is of type "Callable[[Self], Self]", not "Callable[[Foo7[int]], Foo7[int]]" [assert-type]']
"""
11 changes: 4 additions & 7 deletions conformance/results/mypy/overloads_evaluation.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,7 @@ Line 161: Unexpected errors ['overloads_evaluation.py:161: error: No overload va
Line 162: Unexpected errors ['overloads_evaluation.py:162: error: Expression is of type "Any", not "Literal[0, 1]" [assert-type]']
Line 205: Unexpected errors ['overloads_evaluation.py:205: error: Argument 1 to "expand_tuple" has incompatible type "tuple[int, int | str]"; expected "tuple[int, int]" [arg-type]']
Line 206: Unexpected errors ['overloads_evaluation.py:206: error: Expression is of type "int", not "int | str" [assert-type]']
Line 265: Unexpected errors ['overloads_evaluation.py:265: error: Expression is of type "list[Any]", not "Any" [assert-type]']
Line 281: Unexpected errors ['overloads_evaluation.py:281: error: Expression is of type "list[Any]", not "Any" [assert-type]']
Line 303: Unexpected errors ['overloads_evaluation.py:303: error: Expression is of type "Any", not "float" [assert-type]']
Line 347: Unexpected errors ['overloads_evaluation.py:347: error: Expression is of type "list[Any]", not "Any" [assert-type]']
Line 305: Unexpected errors ['overloads_evaluation.py:305: error: Expression is of type "Any", not "float" [assert-type]']
"""
output = """
overloads_evaluation.py:38: error: All overload variants of "example1_1" require at least one argument [call-overload]
Expand Down Expand Up @@ -47,7 +44,7 @@ overloads_evaluation.py:162: error: Expression is of type "Any", not "Literal[0,
overloads_evaluation.py:205: error: Argument 1 to "expand_tuple" has incompatible type "tuple[int, int | str]"; expected "tuple[int, int]" [arg-type]
overloads_evaluation.py:206: error: Expression is of type "int", not "int | str" [assert-type]
overloads_evaluation.py:265: error: Expression is of type "list[Any]", not "Any" [assert-type]
overloads_evaluation.py:281: error: Expression is of type "list[Any]", not "Any" [assert-type]
overloads_evaluation.py:303: error: Expression is of type "Any", not "float" [assert-type]
overloads_evaluation.py:347: error: Expression is of type "list[Any]", not "Any" [assert-type]
overloads_evaluation.py:282: error: Expression is of type "list[Any]", not "Any" [assert-type]
overloads_evaluation.py:305: error: Expression is of type "Any", not "float" [assert-type]
overloads_evaluation.py:349: error: Expression is of type "list[Any]", not "Any" [assert-type]
"""
8 changes: 4 additions & 4 deletions conformance/results/mypy/specialtypes_never.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
conformant = "Pass"
output = """
specialtypes_never.py:19: error: Implicit return in function which does not return [misc]
specialtypes_never.py:86: error: Incompatible types in assignment (expression has type "list[Never]", variable has type "list[int]") [assignment]
specialtypes_never.py:86: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
specialtypes_never.py:86: note: Consider using "Sequence" instead, which is covariant
specialtypes_never.py:105: error: Incompatible return value type (got "ClassC[Never]", expected "ClassC[U]") [return-value]
specialtypes_never.py:87: error: Incompatible types in assignment (expression has type "list[Never]", variable has type "list[int]") [assignment]
specialtypes_never.py:87: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
specialtypes_never.py:87: note: Consider using "Sequence" instead, which is covariant
specialtypes_never.py:106: error: Incompatible return value type (got "ClassC[Never]", expected "ClassC[U]") [return-value]
"""
conformance_automated = "Pass"
errors_diff = """
Expand Down
2 changes: 1 addition & 1 deletion conformance/results/mypy/version.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version = "mypy 1.19.0"
version = "mypy 1.19.1"
8 changes: 1 addition & 7 deletions conformance/results/pyrefly/generics_defaults.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
conformant = "Partial"
notes = """
Defaults are not bound by attribute access
"""
conformance_automated = "Fail"
conformance_automated = "Pass"
errors_diff = """
Line 171: Unexpected errors ['assert_type([DefaultIntT](Foo7[DefaultIntT]) -> Foo7[DefaultIntT], (Foo7[int]) -> Foo7[int]) failed [assert-type]']
"""
output = """
ERROR generics_defaults.py:24:7-31: Type parameter `T` without a default cannot follow type parameter `DefaultStrT` with a default [invalid-type-var]
Expand All @@ -13,5 +8,4 @@ ERROR generics_defaults.py:107:51-54: Expected default `int` of `Invalid1` to be
ERROR generics_defaults.py:114:52-55: Expected default `int` of `Invalid2` to be one of the following constraints: `float`, `str` [invalid-type-var]
ERROR generics_defaults.py:132:12-27: assert_type(int, Any) failed [assert-type]
ERROR generics_defaults.py:143:7-11: TypeVar `T5` with a default cannot follow TypeVarTuple `Ts` [invalid-type-var]
ERROR generics_defaults.py:171:12-57: assert_type([DefaultIntT](Foo7[DefaultIntT]) -> Foo7[DefaultIntT], (Foo7[int]) -> Foo7[int]) failed [assert-type]
"""
3 changes: 3 additions & 0 deletions conformance/results/pyrefly/overloads_evaluation.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,7 @@ ERROR overloads_evaluation.py:38:11-13: No matching overload found for function
ERROR overloads_evaluation.py:46:11-17: No matching overload found for function `example1_1` called with arguments: (Literal[1], Literal[1]) [no-matching-overload]
ERROR overloads_evaluation.py:51:11-14: No matching overload found for function `example1_1` called with arguments: (Literal[1]) [no-matching-overload]
ERROR overloads_evaluation.py:116:13-22: No matching overload found for function `example2` called with arguments: (int | str, int | str, Literal[1]) [no-matching-overload]
ERROR overloads_evaluation.py:266:16-33: assert_type(Any, list[Any]) failed [assert-type]
ERROR overloads_evaluation.py:283:16-40: assert_type(Any, list[Any]) failed [assert-type]
ERROR overloads_evaluation.py:350:16-33: assert_type(Any, list[Any]) failed [assert-type]
"""
9 changes: 2 additions & 7 deletions conformance/results/pyrefly/protocols_explicit.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,11 @@
conformant = "Partial"
notes = """
Does not reject call to super method with no default implementation.
Does not detect stub methods inherited from protocols as abstract.
"""
conformance_automated = "Fail"
conformance_automated = "Pass"
errors_diff = """
Line 89: Expected 1 errors
"""
output = """
ERROR protocols_explicit.py:27:16-28: Method `draw` inherited from class `PColor` has no implementation and cannot be accessed via `super()` [missing-attribute]
ERROR protocols_explicit.py:56:20-36: `tuple[int, int, str]` is not assignable to attribute `rgb` with type `tuple[int, int, int]` [bad-assignment]
ERROR protocols_explicit.py:60:10-20: Cannot instantiate `Point` because the following members are abstract: `intensity`, `transparency` [bad-instantiation]
ERROR protocols_explicit.py:89:15-17: Cannot instantiate `Concrete1` because the following members are abstract: `cm1` [bad-instantiation]
ERROR protocols_explicit.py:134:15-17: Cannot instantiate `Concrete5` because the following members are abstract: `method1` [bad-instantiation]
ERROR protocols_explicit.py:164:17-19: Cannot instantiate `Concrete7A` because the following members are abstract: `method1` [bad-instantiation]
"""
8 changes: 4 additions & 4 deletions conformance/results/pyrefly/specialtypes_never.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ Does not detect invalid return is unreachable
"""
conformance_automated = "Fail"
errors_diff = """
Line 32: Unexpected errors ["Returned type `Literal['whatever works']` is not assignable to declared return type `int` [bad-return]"]
Line 33: Unexpected errors ["Returned type `Literal['whatever works']` is not assignable to declared return type `int` [bad-return]"]
"""
output = """
ERROR specialtypes_never.py:19:22-30: Function declared to return `NoReturn` but is missing an explicit `return` [bad-return]
ERROR specialtypes_never.py:32:12-28: Returned type `Literal['whatever works']` is not assignable to declared return type `int` [bad-return]
ERROR specialtypes_never.py:86:21-22: `list[Never]` is not assignable to `list[int]` [bad-assignment]
ERROR specialtypes_never.py:105:12-27: Returned type `ClassC[Never]` is not assignable to declared return type `ClassC[U]` [bad-return]
ERROR specialtypes_never.py:33:12-28: Returned type `Literal['whatever works']` is not assignable to declared return type `int` [bad-return]
ERROR specialtypes_never.py:87:21-22: `list[Never]` is not assignable to `list[int]` [bad-assignment]
ERROR specialtypes_never.py:106:12-27: Returned type `ClassC[Never]` is not assignable to declared return type `ClassC[U]` [bad-return]
"""
2 changes: 1 addition & 1 deletion conformance/results/pyrefly/version.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version = "pyrefly 0.45.2"
version = "pyrefly 0.46.0"
9 changes: 7 additions & 2 deletions conformance/results/pyright/overloads_evaluation.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ Does not evaluate Any in some cases where overload is ambiguous.
"""
conformance_automated = "Fail"
errors_diff = """
Line 281: Unexpected errors ['overloads_evaluation.py:281:17 - error: "assert_type" mismatch: expected "Any" but received "list[int]" (reportAssertTypeFailure)']
Lines 282, 283: Expected exactly one error (tag 'example5')
Line 282: Unexpected errors ['overloads_evaluation.py:282:17 - error: "assert_type" mismatch: expected "Any" but received "list[int]" (reportAssertTypeFailure)']
Line 283: Unexpected errors ['overloads_evaluation.py:283:17 - error: "assert_type" mismatch: expected "list[Any]" but received "list[int]" (reportAssertTypeFailure)']
"""
output = """
overloads_evaluation.py:38:1 - error: No overloads for "example1_1" match the provided arguments
Expand All @@ -20,5 +22,8 @@ overloads_evaluation.py:116:14 - error: Argument of type "int | str" cannot be a
overloads_evaluation.py:116:17 - error: Argument of type "int | str" cannot be assigned to parameter "y" of type "int" in function "example2"
  Type "int | str" is not assignable to type "int"
    "str" is not assignable to "int" (reportArgumentType)
overloads_evaluation.py:281:17 - error: "assert_type" mismatch: expected "Any" but received "list[int]" (reportAssertTypeFailure)
overloads_evaluation.py:266:17 - error: "assert_type" mismatch: expected "list[Any]" but received "Unknown" (reportAssertTypeFailure)
overloads_evaluation.py:282:17 - error: "assert_type" mismatch: expected "Any" but received "list[int]" (reportAssertTypeFailure)
overloads_evaluation.py:283:17 - error: "assert_type" mismatch: expected "list[Any]" but received "list[int]" (reportAssertTypeFailure)
overloads_evaluation.py:350:17 - error: "assert_type" mismatch: expected "list[Any]" but received "Unknown" (reportAssertTypeFailure)
"""
4 changes: 2 additions & 2 deletions conformance/results/pyright/specialtypes_never.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
conformant = "Pass"
output = """
specialtypes_never.py:19:22 - error: Function with declared return type "NoReturn" cannot return "None" (reportReturnType)
specialtypes_never.py:86:21 - error: Type "list[Never]" is not assignable to declared type "list[int]"
specialtypes_never.py:87:21 - error: Type "list[Never]" is not assignable to declared type "list[int]"
  "list[Never]" is not assignable to "list[int]"
    Type parameter "_T@list" is invariant, but "Never" is not the same as "int"
    Consider switching from "list" to "Sequence" which is covariant (reportAssignmentType)
specialtypes_never.py:105:12 - error: Type "ClassC[Never]" is not assignable to return type "ClassC[U@func10]"
specialtypes_never.py:106:12 - error: Type "ClassC[Never]" is not assignable to return type "ClassC[U@func10]"
  "ClassC[Never]" is not assignable to "ClassC[U@func10]"
    Type parameter "T@ClassC" is invariant, but "Never" is not the same as "U@func10" (reportReturnType)
"""
Expand Down
Loading