Regarding the discussion about using the tilde operator (an idea which has
grown on me a bit):
Would it help, for the purpose of alerting the user to potential backwards
compatibility issues, to add a *second* dunder method, __optional__, or
perhaps __tilde__. to the tilde operator resolution order (is that the
correct phrase...?) that applies to `type` objects only?
Possible additional benefit:
- A FooMcls.__optional__ method could provide ability to use
typing.Optional to type against a custom MissingType
- example: if Foo's metaclass is defined such that Optional[Foo]
produces Union[Foo, MissingType]:
class MissingType: ...
MISSING = MissingType()
def func(x: Optional[Foo] = MISSING): ...
- The Optional type would need to be changed to look for the
__optional__ method at mpy runtime
The logic could be like so:
- when ~foo is invoked...
- if foo has an __invert__ method, tilde always uses that; this will
preserve compatibility for objects
- if foo is a type, use the __optional__ method, if it is present
(this step is skipped for non-type objects)
- if a type object provides *both*, when used in the context of
typing, provide a warning to the user that the Foo type object
is returning
a value from __invert__, which is not as expected
---
Ricky.
"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
On Tue, Sep 3, 2019 at 11:42 AM Philippe Prados <[email protected]> wrote:
> Hello,
>
> I propose to resume all the arguments (I add my remarks in italic) and
> major questions.
>
> Add a new operator for `Union[type1|type2]` ?
>
> -
>
> CONS: This is not a new proposal. If I recall correctly, it was
> proposed way back at the very beginning of the type-hinting discussion, and
> there has been at least one closed feature request for it:
> https://github.com/python/typing/issues/387
> -
>
> It is maybe too late to change this, many people are already get
> used to current notation.
> -
>
> I propose to add a new notation, not to replace the notation
> -
>
> This syntax is difficult to google, if someone encounters it in code
> -
>
> It is still not possible to use `|` for unions because of built-in
> types. (This would require a corresponding slot in type which is a
> non-starter)
> -
>
> I do it
> -
>
> There are currently no volunteer to implement this in mypy
> -
>
> I implement this (One patch for CPython
> <https://github.com/pprados/cpython> and one for MyPy
> <https://github.com/pprados/mypy>).
> -
>
> “but as @ilevkivskyi pointed out, that is not an option (at least
> until Python 4).”
> -
>
> Is it time now ?
> -
>
> PRO: It’s similar of Scala
> <https://dotty.epfl.ch/docs/reference/new-types/union-types.html>
> -
>
> PRO: Seems like `foo | None` is just as readable
> -
>
> PRO: Which means you couldn’t use this feature in Python 3.7, much
> less 2.7. I’m not sure it maintaining backward compatibility in typing and
> in mypy is still as important today as it was 5 years ago, but I’m pretty
> sure it hasn’t been abandoned entirely.
> -
>
> CONS: add operator introducing a dependency to typing in builtins
> -
>
> CONS: supporting this would likely break compatibility with existing
> code that overloads `|` for class objects using a metaclass. We could
> perhaps work around this by making `|` inside an annotation context
> different from the regular `|` operator.
> -
>
> A work around is to use `Union[type1,type2]` in this case
> -
>
> CONS: as breaking the backport (in that typing.py can easily be
> backported but core `types` can't)
> -
>
> There are several things in the typing syntax that require a
> certain minimum version. E.g. type annotations require Python 3 (whereas
> type comments work in Python 2 too), type annotations on variables (PEP
> 526) require 3.6+, `from __future__ import annotations` (PEP 563)
> requires 3.7+.
> -
>
> PRO: I mean that at run-time, `int|str` might return a very simple
> object in 3.9, rather than everything that you'd need to grab from
> importing `typing`. Wondering if doing so would close off the
> possibility of, in 3.12 or something, making it a more directly usable
> "type union" that has other value.
> -
>
> CONS: if Python itself doesn't have to be changed, we'd still need to
> implement it in mypy, Pyre, PyCharm, Pytype, and who knows what else.
> -
>
> My patch of mypy is just 20 lines of codes
>
> If yes,
>
> -
>
> Change only the PEP484 <https://www.python.org/dev/peps/pep-0484/>
> (Type hints) to accept the syntax `type1 | type2` ?
> -
>
> PRO: The PEP563 <https://www.python.org/dev/peps/pep-0563/>
> (Postponed Evaluation of Annotations) is enough to accept this
> proposition
> -
>
> CONS: The Resolving type hints at runtime
>
> <https://www.python.org/dev/peps/pep-0563/#resolving-type-hints-at-runtime>
> says: “For code which uses annotations for other purposes, a
> regular eval(ann, globals, locals) call is enough to resolve the
> annotation.”.
> Without add a new operator `__or__` in type `type`, it’s not
> possible to resolve type hints at runtime.
>
> >>> from __future__ import annotations
> >>> def foo() -> int | str: pass
> ...
> >>> eval(foo.__annotations__['return'])
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "<string>", line 1, in <module>
> TypeError: unsupported operand type(s) for |: 'type' and 'type'
>
> -
>
> CONS: Without operator, it’s not possible to write
>
> >>> u = int | str
> >>> u
> typing.Union[int, str]
>
>
> -
>
> Use `(int, str)` in place of `Union[int,str]` ?
> -
>
> PRO: This doesn't have compatibility issues and it's similar to
> `isinstance(foo,
> (int, str))`
> -
>
> PRO: Either better than breaking backward compatibility by adding
> new operator methods to the type `type`.
> -
>
> CONS: In most languages with similar-ish type syntax, `(int, str)`
> means `Tuple[int, str]`, not `Union[int, str]`.
> -
>
> Use `{int, str}` in place of `Union[int,str]` ?
> -
>
> PRO: big advantage of `{int, str}` over `int|str`. It doesn't
> require adding anything to `type`, and we don't need to introduce a
> new lightweight builtin union type.
>
> Add a new operator for `Optional[type]` ?
>
> -
>
> CONS: `foo | None` is short and readable
> -
>
> CONS: `foo | None` it's 3 fewer characters than `Optional[foo]`, or 30
> fewer if you include the full removal of `from typing import Optional`.
> the additional gain of `~foo` is only 6 characters.
> -
>
> PRO: help the readability, with a lot of parameters:
>
> def f(source: str | None, destination: str | None, param: int | None):...
> def f(source: ~str, destination: ~str, param: ~int):...
>
> -
>
> PRO: I'm currently working on annotating a very large codebase, and
> `Optional[T]` is so frequent that I think `T | None` would not be
> enough of an improvement.
> -
>
> PRO: Adding a default `__or__` overload to `type` seems a reasonable
> price to pay in 3.9, and ditto for `__invert__`. Type checkers can
> support this in older Python versions using PEP 563 or in type comments or
> in "forward references" (types hidden in string literals).
> -
>
> CONS: The `~` is easy to be missed (at least by human readers) and the
> meaning not obvious.
> -
>
> PRO: Also, Python’s typing system is a lot easier to grasp if you’re
> familiar with an established modern-typed language (Swift, Scala, Haskell,
> F#, etc.), and they also use `Optional[T]` (or `optional<T>` or `Maybe
> t` or some other spelling of the same idea) all over be place—so often
> that many of them have added shortcuts like `T?` to make it easier to
> write and less intrusive to read.
>
> if yes,
>
> Add operator `__revert__` in type type to use syntax like `~int` ?
>
> -
>
> CONS: `~` is not automatically readable
> -
>
> like `:` to separate variable and typing.
> -
>
> CONS: `~` means complement, which is a completely different thing from
> `|None`. `~int` seems like it would actually harm comprehension
> instead of helping.
> -
>
> PRO: the slight abuse of `~int` meaning "maybe int" is pretty
> plausible (consider how "approximately equal" is written mathematically).
> -
>
> PRO: Possibly relevant for tilde:
>
> https://www.thecut.com/article/why-the-internet-tilde-is-our-most-perfect-tool-for-snark.html
>
> -
>
> CONS: With `~` there probably won't be a confusion in that sense, but
> someone reading it for the first time will definitely need to look it up
> (which is fine i.m.o.).
> -
>
> Like the first time someone reading the annotation
>
> def f(a=int):...
> def f(a:int):...
>
> Add operator __add__ in type type to use syntax like +int ?
>
> -
>
> PRO: `+foo` definitely seems to say "foo, plus something else" to me
> much more than `~foo`.
> -
>
> CON: `+foo` is less intuitive than `~foo` for `Optional`
>
> Like Kotlin <https://kotlinlang.org/docs/reference/null-safety.html>, add
> a new `?` operator to use syntax like `int?` ou `?int` ?
>
> -
>
> CONS: It’s not compatible with IPython and Jupyter Lab `?smth`
> displays help for symbol `smth`
> -
>
> CONS: With default arguments, `?=` looks... not great
> -
>
> def f(source: str?=def_src, destination: str?=MISSING, param: int?=1):
> ...
>
> Extend `isinstance()` and `issubclass()` to accept `Union` ?
>
> isinstance(x, str | int) ==> "is x an instance of str or int"
>
> -
>
> PRO: if they were permitted, then instance checks could use an
> extremely clean-looking notation for "any of these":
>
> Do nothing, open a new PEP or extend PEP585 ?
>
> -
>
> I do think we should probably review PEP 585
> <https://www.python.org/dev/peps/pep-0585/> before doing anything
> about unions specifically -- likely there are bigger fish to fry
> -
>
> PEP 585 has not received much discussion
>
>
> So, I think it’s time to answer these questions:
>
> -
>
> Add a new operator for `Union[type1|type2]` ?
> -
>
> If yes
> -
>
> Change only the PEP484 (Type hints) to accept the syntax `type1 |
> type2` ?
> -
>
> Use `(int, str)` in place of `Union[int,str]` ?
> -
>
> Use `{int, str]` in place of `Union[int,str]` ?
> -
>
> Add a new operator for `Optional[type]` ?
> -
>
> If yes,
> -
>
> Add operator `__revert__` in type type to use syntax like `~int` ?
> -
>
> Add operator `__add__` in type type to use syntax like +int ?
> -
>
> Extend `isinstance()` and `issubclass()` to accept `Union` ?
>
> Do nothing, open a new PEP
> <https://github.com/pprados/peps/blob/master/pep-9999.rst> or extend
> PEP585 <https://www.python.org/dev/peps/pep-0585/> ?
>
> Philippe
>
> Le mar. 3 sept. 2019 à 08:30, Philippe Prados <[email protected]> a écrit :
>
>> With my implementation, I can check
>> assert int | None == None | int
>> is true
>>
>> Le lun. 2 sept. 2019 à 13:32, Ivan Levkivskyi <[email protected]> a
>> écrit :
>>
>>> On Thu, 29 Aug 2019 at 23:48, Guido van Rossum <[email protected]> wrote:
>>>
>>>> On Thu, Aug 29, 2019 at 3:33 PM Chris Angelico <[email protected]>
>>>> wrote:
>>>>
>>>>> On Fri, Aug 30, 2019 at 8:28 AM Guido van Rossum <[email protected]>
>>>>> wrote:
>>>>
>>>> [...]
>>>>
>>>>
>>>> I do tink we should probably review PEP 585 before doing anything about
>>>> unions specifically -- likely there are bigger fish to fry. (And PEP 585
>>>> has not received much discussion.)
>>>>
>>>
>>> I also agree with this. Generally I am fine with Union[int, str] and
>>> Optional[int], but I also see how some people might want a shorter
>>> notation. Many things around typing have been previously
>>> rejected because we didn't want to introduce any (or at least minimal)
>>> changes to the syntax and runtime, but now that typing is much more widely
>>> used we can reconsider some of these.
>>> Importantly, I think this should be done in a systematic way
>>> (potentially using PEP 585 draft as a starting point).
>>>
>>> --
>>> Ivan
>>>
>>> _______________________________________________
>>> Python-ideas mailing list -- [email protected]
>>> To unsubscribe send an email to [email protected]
>>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/[email protected]/message/YZHAERD5T4TZL62A6EI4BCKKAQYNJGSU/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> _______________________________________________
> Python-ideas mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/SAHJWREC7ESRH3WTDITRM2JSI3OUMPFM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/KCTOMMFGSLZDA6ZY6MGJM2IJWWTFI2RB/
Code of Conduct: http://python.org/psf/codeofconduct/
