Thursday, 2 March 2017

Mypy 0.501 Released

We’ve just uploaded mypy 0.501 to PyPI. This release adds new features, bug fixes and library stub updates. You can install it as follows:

    python3 -m pip install --upgrade mypy

Note: Remember that the package name is now mypy (no longer mypy-lang).

Switch to New Parser

Mypy now uses the typed_ast parser by default, which is more correct and significantly faster. The old parser is available for now with --no-fast-parser, but we are planning to remove it in the next release. Previously you had to enable the new parser explicitly through --fast-parser.

Note that mypy 0.501 requires a new, backwards incompatible version of typed_ast (1.0.1 or higher). This should be taken care of automatically by the package dependencies if you install mypy using pip .

Default to Python 3.6 Mode

Supported Python 3.6 features are now enabled by default, such as variable annotations (PEP 526), f-strings (PEP 498), and underscores in numeric literals (PEP 515).

Simple Metaclasses

Mypy now supports simple metaclasses. Mypy can look up attributes and methods defined in the metaclass, but mypy still doesn’t understand more dynamic metaclass behavior. For example, this works now:

    class Meta(type):
        def __add__(self, x: str) -> str:
            return 'a' + x

    class C(metaclass=Meta):
        ...

    print(C + 'x')  # Okay

You can also use metaclasses in Python 2:

    ...

    class C(object):
        __metaclass__ = Meta

This was contributed by Elazar Gershuni with help from Tom Manderson.

Warn about Missing Return Statements by Default

The --warn-no-return flag is now on by default, which means mypy will warn about functions missing a return statement on one or more code paths, unless the function’s return type is None or Any. For example:

        def f(n: int) -> int:  # Missing return statement
            if n > 3:
                return n + 1

Mypy currently doesn’t warn about empty function bodies or bodies that only have an ellipsis (…), since these are sometimes used for abstract methods and aren’t necessarily a problem. The option can generate false positives if you rely on the implicit None return value when execution falls off the end of a function. This behavior can be disabled globally with --no-warn-no-return or on a per-module basis with the warn_no_return config option.

This was contributed by Reid Barton.

NoReturn

You can now annotate functions that never return using mypy_extensions.NoReturn. Mypy will check that those functions do, in fact, not return, and will understand that they cause code paths to never return when called. Mypy now also knows that certain library functions such as sys.exit() don’t return. For example:

    from mypy_extensions import NoReturn
    def bad_number() -> NoReturn:
        raise RuntimeError("Bad number.")

    def f(n: int) -> int:  # okay
        if n > 3:
            return n + 1
        else:
            bad_number()

Install mypy_extensions using pip to use NoReturn:

    python3 -m pip install --upgrade mypy-extensions  # For Python 3
    pip install --upgrade mypy-extensions  # For Python 2

Leaving Out Redundant Generic[T]

You can omit redundant Generic[T] in the base class list if you have another reference to T within base classes. For example, this now works:

    from typing import TypeVar, Iterable

    T = TypeVar('T')

    class MyCollection(Iterable[T]):
        ...
        def __iter__(self) -> Iterator[T]: ...

    def collection_to_list(c: MyCollection[int]) -> List[int]:
        return list(c)

For previous versions of mypy you would have needed to write the class definition like this:

    ...
    class MyCollection(Iterable[T], Generic[T]):
        ...

This was contributed by Ivan Levkivskyi.

Better Checking of Assignments in Class Body

Mypy finally can check if assignments within a class body are compatible with base classes. Now this will get flagged as an error:

    class A:
        x = 1

    class B(A):
        x = 'a'  # Error

This was contributed by TrueBrain.

Strict Mode

Mypy now has a strict mode which turns on most flags that add additional checks. Strict mode can be enabled with --strict. Some of these checks may be opinionated (like --strict-boolean). New checks will be added to strict mode over time.

Invertible Options

Most boolean flags can now be explicitly inverted. See mypy --help for the names of the inverted options. For example, the inverse of --disallow-untyped-defs is --allow-untyped-defs. This is useful when an option is enabled in the config file but you want to disable it for a specific run without editing the config file.

More New Features

  • Read configuration from setup.cfg if mypy.ini not present. (Dominik Miedziński)
  • Support type comments for with and for statements. (Alex Frieder)
  • Support type comments for async with and async for statements. (Alex Frieder)
  • Support E[<str>] when E is an Enum type.
  • Support @no_type_check in fast parser.
  • api.run now accepts a list of command-line arguments and returns a 3-tuple, including exit status. (Jacques de Hooge)
  • Add --strict-boolean flag that warns about non-bool values used as conditions. (Alex Frieder)
  • Add --warn-return-any flag that warns about returning a value with Any type. (lincolnq)
  • Add support for __getattribute__. (Jelle Zijlstra)
  • Issue warning for possibly malformed asserts. (Alex Frieder)
  • Make most fast parser errors non-fatal.
  • Improve fast parser error messages.
  • TypedDict improvements (TypedDict is still undocumented and experimental). (Ivan Levkivskyi)

Notable Bugs Fixed

  • Detect missing imports of List, Dict and Set. (Ivan Levkivskyi)
  • Don't complain about method override when the base class method has type Any. (Naomi Seyfer)
  • Fix stubgen crash on an extension module containing an alias of object. (Jelle Zijlstra)
  • Fix crash on access to local classes. (Ivan Levkivskyi)
  • Fix crash on expressions like "%d%d" % (*x,).
  • Generate errors on classmethod and staticmethod override conflicts. (Alex Frieder)
  • Fix crash on invalid type comments/annotations. (Ivan Levkivskyi)
  • Fix crash when assigning to a variable with a partial type. (Alex Frieder)
  • Treat varargs as valid context in default lambda argument. (Elazar Gershuni)
  • Always process modules/packages given with -m /-p.
  • Generate error when using deleted variable in a conditional. (Alex Frieder)
  • Make fast parser able to parse machine-generated code with deep nesting. (Łukasz Langa)
  • Fix bug with type variables defined in different scopes being treated as the same type variable. (with help from Ivan Levkivskyi)
  • Generate error on Any(...) which fails at runtime. (Alex Frieder)
  • Check for duplicate names in function definitions when using the fast parser. (Alex Frieder)
  • Disallow duplicate type comments/annotations in fast parser. (Alex Frieder)
  • Fix error context for indexed assignment with an annotation. (Jelle Zijlstra)
  • Fix return type of dict.get in strict optional mode.
  • Type check assert messages in fast parser. (Alex Frieder)
  • Fix warning about missing return statements in async def functions. (Jelle Zijlstra)
  • Order of union items no longer affects type sameness. (Alex Frieder)
  • Propagate inference type context through reveal_type for more precise revealed types.
  • Allow unicode docstrings in Python 2. (Jelle Zijlstra)
  • Fix several Python 2 bugs in fast parser.
  • Fix some additional crashes.

Other Changes

Acknowledgments

Thanks to all mypy contributors who contributed to this release:

  • Adrien Chauve
  • Alex Frieder
  • Brandon W Maister
  • Dominik Miedziński
  • Eduard-Cristian Stefan
  • Elazar Gershuni
  • Ivan Levkivskyi
  • Jacques de Hooge
  • Jelle Zijlstra
  • lincolnq
  • Łukasz Langa
  • Naomi Seyfer
  • Tom Manderson
  • TrueBrain
  • Wojciech Kaczmarek

Additional thanks to these contributors to typeshed:

  • 5j9
  • Adam Marszałek
  • Alex Frieder
  • Alexey (forestbiiird)
  • Andrey Vlasovskikh
  • aostiles
  • Avy Faingezicht
  • Bertrand Bonnefoy-Claudet
  • David Euresti
  • Dominik Miedziński
  • Dylan Jenkinson
  • Eklavya Sharma
  • Eric Moyer
  • Ethan (ethanhs)
  • Ivan Levkivskyi
  • Jelle Zijlstra
  • lincolnq
  • Lucas Wiman
  • Łukasz Langa
  • Matt Kimball
  • Matthias Kramm
  • Mike Patek
  • Mohab Usama
  • rchen152
  • Richard Hansen
  • Roy Williams
  • Russ Allbery
  • Sunghyun Hwang
  • syrrim
  • Tim Abbott
  • Tom Manderson
  • Tomasz Elendt
  • Valérian Rousset

— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)

Friday, 13 January 2017

Mypy 0.470 Released

We’ve just uploaded mypy 0.470 to PyPI. This release adds new features, bug fixes and library stub updates. You can install it as follows:

    python3 -m pip install mypy

Note: The package name is now “mypy” (no longer mypy-lang). If you have an older mypy version installed, remove it first, before installing the new package:

    python3 -m pip uninstall mypy-lang

Good news: New Package Name and Version Style

We have finally obtained ownership of the “mypy” package name on PyPI. In order to avoid conflicts with (hypothetical) users of the previous occupant of the “mypy” package name we’re changing the style of version numbers we’re using (at least until 1.0 comes along) — the new version is now 0.470. If you are using requirements.txt files, the proper incantation is now:

    mypy==0.470

The old “mypy-lang” package will no longer be upgraded (the last version there is 0.4.6).

New Import Handling Options

The “silent imports” functionality, useful to avoid noisy errors about missing modules when annotating a large codebase, has been refactored and improved. There are now two separate flags (see the online docs for the full scoop):

  • --follow-imports=arg takes an argument which must be one of the following:
    • normal — the default behavior
    • silent — tries to find, parse and check imported modules (same as normal) but suppresses all errors for those modules that were not given on the command line
    • skip — roughly equivalent to the old --silent-imports flag
    • error — roughly equivalent to the old --almost-silent flag
  • --ignore-missing-imports suppresses error messages about imports that cannot be resolved at all. This was previously implied by --silent-imports unless --almost-silent was given.

Fast Parser Supported on Windows

The --fast-parser option now works also on Windows. It’s faster than the default parser and supports more Python syntax, including new syntax introduced in Python 3.6. The required package typed_ast is now installed by default as a mypy dependency (see below for more). We are planning to deprecate the current default parser in a future mypy release. (Also, the mypy project now runs tests for Windows on Appveyor.)

Improved Type Inference for lambda Expressions

A lambda expression without type context is now inferred as a Callable returning whatever type can be inferred for the expression in the body of the lambda. Previously such lambda expressions were given type Any . Note that lambda ...: None is now considered to have type Callable[..., None] which cannot be called to provide a value, only as a procedure call. The arguments are given type Any, so that e.g. the following two are now inferring the same type for a, i.e. Callable[[Any], List[Any]]:

    a = lambda x: [x]

    def a(x) -> List[Any]: return [x]

(Contributed by Elazar Gershuni)

Support for callable()

A condition using e.g. if callable(x) now causes mypy to infer a Callable type for x in the block controlled by the condition, and a non-Callable type in the block controlled by its negation. For example, this now type-checks:

    def maybecall(x: Union[int, Callable[[], int]]) -> int:
        if callable(x):
            return x()
        else:
            return x

(Contributed by Alex Frieder)

New Package Dependencies

Mypy now has automatic dependencies on typing and typed_ast. Previously typed_ast had to be manually installed using pip and typing was bundled with the mypy package.

Note that typed_ast requires Python 3.5+ on Windows. We’re dropping support for running mypy on Windows using Python 3.4 or older. (But checking still supports 2.7 and 3.2+ targets, using the --python-version flag.)

Notable Bugs Fixed

  • Callable type compatibility checking rewritten with several improvements (Naomi Seyfer)
  • Fix Callable arguments in overloaded functions
  • Fix lambda inference with simple Union contexts
  • Fix spurious warnings caused by type redeclarations (Łukasz Langa)
  • Fix handling of empty tuple type aliases
  • Fix crash on retrieving TypeVar from class (Tom Manderson)
  • Ensure required keyword-only arguments are provided (Naomi Seyfer)
  • Consider types like Tuple[t1, t2, ...] more consistently as subtypes of plain tuple (Ivan Levkivskyi)
  • Compare cached options to current per-file options when using --incremental
  • Fix NamedTuple defined in a method

New Experimental Features

There is work-in-progress support for dictionaries with literal string keys (TypedDict). Different keys can have items with different types, and mypy will figure out the right type for each key. For example, it will be possible to give a precise type for this dictionary:

    d = {'path': 'file.txt', 'size': 1254}

TypedDict is still very tentative and there’s no official documentation yet. You need to import TypedDict from mypy_extensions; use pip install -U mypy_extensions to install it. (Contributed by David Foster)

Descriptors are now supported. We're still not quite sure how well they work in practice. (Contributed by Calen Pennington)

Other Changes in This Release

  • Turn --hide-error-context on by default to generate more concise output from mypy. Add --show-error-context to re-enable the old behavior.
  • Arguments starting with __ (two underscores) are now positional-only. (Naomi Seyfer)
  • Add an API for calling mypy directly from another python application. (Jacques de Hooge)
  • Add ignore_errors config option to selectively ignore all non-fatal errors in some files.
  • Add Python 3 cheat sheet. (Ethan Smith)
  • Speed up the mypy test suite. (Łukasz Langa)
  • Miscellaneous other bug fixes.
  • Many updates to the library stubs in typeshed.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Aleksander Vognild Burkow
  • Alex Frieder
  • Bertrand Bonnefoy-Claudet
  • Calen Pennington
  • Chris Oelmueller
  • David Euresti
  • David Foster
  • Elazar Gershuni
  • Ethan (ethanhs)
  • Ivan Levkivskyi
  • Jacques de Hooge
  • Jakub Stasiak
  • Jelle Zijlstra
  • Josiah Boning
  • Juanvulcano
  • Łukasz Langa
  • Naomi Seyfer
  • Roy Williams
  • Tetsuya Morimoto
  • Tom Manderson
  • TrueBrain

Additional thanks to these contributors to typeshed:

  • Alex Frieder
  • Alex Jurkiewicz
  • Anders Kaseorg
  • Bertrand Bonnefoy-Claudet
  • Cadel Watson
  • Calen Pennington
  • Daisuke Miyakawa
  • Danny Weinberg
  • David Euresti
  • Eric Moyer
  • George King
  • gotyaoi
  • Henri Dwyer
  • Hugo (hugovk)
  • Jason Fried
  • Jelle Zijlstra
  • Jon Dufresne
  • Joseph H Garvin
  • Josiah Boning
  • Kosaka Masayuki
  • lionel-github
  • Luiz Menezesf
  • Łukasz Langa
  • Madeleine Thompson
  • Mateusz Kurek
  • Matthias Kramm
  • Michael Lee
  • Mohab Usama
  • Naomi Seyfer
  • Nicolas Duchastel de Montrouge
  • Onno Kortmann
  • Peter Amstutz
  • Philip House
  • Reverb Chu
  • Richard Eames
  • Roy Williams
  • Ryan C. Thompson
  • rymdhund
  • Simon Ekstrand
  • Thomas Aynaud
  • Thomas Cellerier
  • Tom Manderson
  • TrueBrain
  • Wesley Bowman
  • z33ky

— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)