[Python-Dev] Re: Should python installation modify /etc/paths.d on Mac OS X?

2021-01-18 Thread Barry Scott 



> On 14 Jan 2021, at 14:22, Peng Yu  wrote:
> 
> Hi,
> 
> The /etc/paths.d directory on Mac OS X contains the following
> directory. Some packages have their PATH variables set there.
> 
> For Python, I have to set the PATH manually in my ~/.bashrc. Should
> the Python developers consider setting PATH in /etc/paths.d as well?
> 
> ==> /etc/paths.d/100-rvictl <==
> /Library/Apple/usr/bin
> 
> ==> /etc/paths.d/40-XQuartz <==
> /opt/X11/bin
> 
> ==> /etc/paths.d/com.vmware.fusion.public <==
> /Applications/VMware Fusion.app/Contents/Public

One of the things that I wish the python macOS installed would stop doing is
editing my .bashrc/.zshrc etc.

After each python install I have to undo the edits to .bashrc or .zshrc that 
break
my environment.

I understand the motivation to do this. But please provide a why to turn this 
off
for for people like me that have many versions of python installed.

I think of two things that could be checked for.
1. A marked in the .bashrc/.zshrc file. '# python-disable-path-edit'
2. A file in ~/Library somewhere that disabled the behavour

Barry



> 
> -- 
> Regards,
> Peng
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/[email protected]/message/PHPM33FMHXUIA3PKOETKKX3IXZHOMPP5/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/WD4LS4LCRUC3G6O3PWOAK5NXL3MO3IGB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Unification of the Mac builds?

2021-01-18 Thread Chris Barker via Python-Dev 
Thanks Ronald,

This  is really helpful.

-CHB

On Fri, Jan 15, 2021 at 5:43 AM Ronald Oussoren 
wrote:

>
>
> On 14 Jan 2021, at 23:03, Chris Barker via Python-Dev <
> [email protected]> wrote:
>
> Ned,
>
> Thanks -- I'll take further discussion to the python-mac list.
>
> Ronald:
>
> That’s a feature of the framework build. The unix build is exactly the
>> same as a unix build on other platform.  Adding the same feature to the
>> unix build should be possible, but would complicate the build.  I have no
>> interest to work on this, but would be willing to review PRs,
>>
>
> fair enough.
>
>
>> as long as those aim for feature parity with the framework build. That
>> is, both pythonw(1) and python(1) should redirect through an embedded app
>> bundle.
>>
>
> yes, that's what I'm thinking.
>
> Done, just configure “—enable-universalsdk
>> —with-universal-archs=universal2” without specifying a framework build.
>>
>
> Excellent, thanks!
>
> Something I would like to work on, but don’t have enough free time for, is
>> an alternative installation with an application bundle on the top level
>> instead of a framework.  Installation would then entail dropping “Python
>> X.Y.app” into your application folder, and uninstallation would be to drop
>> the same bundle into the bin.
>>
>
> That would be nice -- and would, I think, actually buy us something useful
> from this bundling :-)
>
>
> To be honest I’d expect that this will also lead to complaining about how
> this build is not a unix build ;-)
>
>
> Just don’t use conda ;-).  To be blunt, doing this properly is trivial.
>>
>
> which "this" are you referring to -- and if it's trivial, then why hasn't
> anyone figured out how in multiple conversations over years?
>
> “This” is having a pythonw that’s the same as in framework builds.
>
> I don’t know. I do know that nobody asked questions about this before.
>
>
> But if it is -- tell us how and maybe we will do it (if it's really
> trivial, maybe even I, with my limited build skills could do it)
>
>
>
> For anyone reading along that isn’t familiar with macOS: On macOS a number
> of system libraries, and in particular GUI libraries, require that they are
> used from an application bundle (‘.app”).   For Python we want to be able
> to just use the command-line for simple scripts (and testing of more
> complicated applications), instead having to build an app bundle every time.
>
> In framework builds this is solved by two changes relative to a unix build:
> 1. The regular python binary is in
> “{sys.prefix}/Resources/Python.app/Contents/MacOS/Python”
> 2. “{sys.executable}” (or “{sys.prefix}/bin/python3”) is a stub executable
> that does nothing but execv[1] the regular python binary. The source of
> this binary is in Mac/Tools/pythonw.c
>
>
> Note: All of this requires using a shared library build (—enable-shared)
> because of the way the stub finds sys.prefix.
>
> To add this to the unix build as well requires:
> 1. Make up a location for the Python.app
> 2. Change the Makefile(.pre.in) to build the stub executable install
> files in their new location, but only when installing on macOS with an
> —enable-shared build
> 3. Tweak the code in pythonw.c to calculate the correct path to Python.app
> 4. Possibly tweak the calculation of sys.prefix for unix builds, IIRC the
> special macOS sauce is enabled for framework builds only
> 5. Do some debugging because this list is likely incomplete, I typed it
> from memory
>
>
> Ronald
>
> [1] for the pedantically correct: it actually uses posix_spawn in exec
> mode to ensure that the correct CPU architecture is used. That’s why “arch
> -x86_64 python3.9” works when you use the universal2 installer for python
> 3.9, if we’d use execv the stub would launch the actual interpreter in
> native mode.
>
> Ronald
>
>
> Thanks,
>
> -CHB
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> [email protected]
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/7YZ4JB5PTL2LKGRCNZF7X4TP7ZYTBW3H/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> —
>
> Twitter / micro.blog: @ronaldoussoren
> Blog: https://blog.ronaldoussoren.net/
>
>

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

[email protected]
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to python-dev-le...@

[Python-Dev] Re: Should python installation modify /etc/paths.d on Mac OS X?

2021-01-18 Thread Ned Deily 
On Jan 18, 2021, at 08:24, Barry Scott  wrote:
> One of the things that I wish the python macOS installed would stop doing is
> editing my .bashrc/.zshrc etc.
> 
> After each python install I have to undo the edits to .bashrc or .zshrc that 
> break
> my environment.
> 
> I understand the motivation to do this. But please provide a why to turn this 
> off
> for for people like me that have many versions of python installed.

As discussed elsewhere, we are looking at other ways to manage this.  However, 
there is already an option to turn this off.  When installing a python.org 
Python pkg from the macOS Installer app, at the fifth step, "Installation 
Type", there is a "Customize" button; clicking on that brings up a list of the 
installer's packages including "Shell profile updater"; just uncheck that 
before proceeding.  If you are using the command line "installer" program, you 
can use its -showChoicesXML option to get the .pkg choices and modify it to 
disable the profile changes package.

--
  Ned Deily
  [email protected] -- []
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/PA47EBL45SZJ2JYFLR7XVUELC6ZPJ6GM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-18 Thread Guido van Rossum 
On Sat, Jan 16, 2021 at 3:32 PM Larry Hastings  wrote:

> [...] If anybody distinguished between "annotations are unset" and
> "annotations are set to an empty dict", that code would fail, but I expect
> nobody ever does that.
>

I agree, since I can't think of differing semantics. Given that
`__annotations__` is filled from annotated class variables, the only reason
someone might care about the difference would be if they are aware of code
that manually *sets* `X.__annotations__ = {}` and they have some kind of
shared understanding that that means something special. I find that highly
unlikely, and frankly, if someone needs such a shared understanding, let
them just pick a unique key to set.

I do worry about the best practice getting worse if your PEP 649 is
accepted.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/PXMLOE6CLYOFFQCCJ5AWHSRE5GAMDWZC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-18 Thread Guido van Rossum 
On Sun, Jan 17, 2021 at 7:33 AM Larry Hastings  wrote:

> If your imports are complicated, you could always hide them in a
> function.  I just tried this and it seems to work fine:
>
> def my_imports():
> global other_mod
> import other_mod
>
> So, you could put all your imports in such a function, run it from inside
> a "if typing.TYPE_CHECKING" block, and you'd have a convenient way of doing
> all your imports from inside IPython too.
>

But static type checkers won't understand such imports. (Or is this about
annotations used for other purposes? Then I suppose it's fine, but only as
long as you completely give up static type checks for modules that use this
idiom.)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/3I7UDD5VAKQJDFW22OKCEPV3G4UPKMZB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Advantages of pattern matching - a simple comparative analysis

2021-01-18 Thread Chris Barker via Python-Dev 
On Mon, Nov 23, 2020 at 8:20 AM Brian Coleman 
wrote:

> Take as an example a function designed to process a tree of nodes similar
> to that which might be output by a JSON parser. There are 4 types of node:
>
> - A node representing JSON strings
> - A node representing JSON numbers
> - A node representing JSON arrays
> - A node representing JSON dictionaries
>
> The function transforms a tree of nodes, beginning at the root node, and
> proceeding recursively through each child node in turn. The result is a
> Python object, with the following transformation applied to each node type:
>
> - A JSON string `->` Python `str`
> - A JSON number `->` Python `float`
> - A JSON array `->` Python `list`
> - A JSON dictionary `->` Python `dict`
>
> I have implemented this function using 3 different approaches:
>
> - The visitor pattern
> - `isinstance` checks against the node type
> - Pattern matching
>

I've always thought that the alternative to a "switch case" construct in
Python (and I suppose most OO languages) is subclassing and method
overriding. I guess that's what the "visitor pattern" is here, but it seems
to be adding a bunch of unnecessary bolierplate.

So: given that you have a special "Node" object anyway, the thing to do is
to have those Node object know how to unpack themselves. Then the top
"traverse the tree" function becomes a single method or attribute access:

tree = node_tree.value

here's what the Nodes look like in this example:

class Node:
def __init__(self, val):
self._value = val

@property
def value(self):
return self._value


class StringNode(Node):
pass


class NumberNode(Node):
pass


class ListNode(Node):
@property
def value(self):
return [item.value for item in self._value]


class DictNode(Node):
@property
def value(self):
return {k: item.value for k, item in self._value.items()}

Of course, this requires that you have control of the Node objects, rather
than getting them from some other library -- but that seems to be what all
the examples here are anyway.

If you do need to parse out a tree of object that are not "special"
already, then you need to do some type of pattern matching / isinstance
checking. In this case, I wrote a function that builds up a tree of Nodes
from arbitrary Python objects:

def make_nodes_from_obj(obj):
if isinstance(obj, str):
return StringNode(obj)
if isinstance(obj, Real):
return NumberNode(obj)
if isinstance(obj, Sequence):
return ListNode([make_nodes_from_obj(item) for item in obj])
if isinstance(obj, Mapping):
return DictNode({k: make_nodes_from_obj(item)
 for k, item in obj.items()})

And that could benefit from pattern matching, I suppose, though it's not
very compelling to me.

And in "real world" code, I've done just this -- building a system for
saving / restoring dataclasses to/from JSON. In that case, each of the
dataclasses knows how to save itself and build itself from JSON-compatible
python objects (numbers, dicts, strings, lists) -- so again, no need for
pattern matching there either. And what I really like about the approach of
putting all the logic in the "nodes" is that I can make new types of nodes
without having to touch the code at the "top" that visits those nodes.

In short -- I'm still looking for a more compelling example :-)

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

[email protected]
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/S6Y24VDUD5RL2SDU3LE56P3AONSAGMGM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-18 Thread Larry Hastings 


On 1/18/21 12:24 PM, Guido van Rossum wrote:
On Sun, Jan 17, 2021 at 7:33 AM Larry Hastings > wrote:


If your imports are complicated, you could always hide them in a
function.  I just tried this and it seems to work fine:

def my_imports():
    global other_mod
    import other_mod

So, you could put all your imports in such a function, run it from
inside a "if typing.TYPE_CHECKING" block, and you'd have a
convenient way of doing all your imports from inside IPython too.


But static type checkers won't understand such imports. (Or is this 
about annotations used for other purposes? Then I suppose it's fine, 
but only as long as you completely give up static type checks for 
modules that use this idiom.)



Oh, okay.  I haven't used the static type checkers, so it's not clear to 
me what powers they do and don't have.  It was only a minor suggestion 
anyway.  Perhaps PEP 649 will be slightly inconvenient to people 
exploring their code inside IPython.


Or maybe it'd work if they gated the if statement on running in ipython?

   if typing.TYPE_CHECKING or os.path.split(sys.argv[0])[1] == "ipython3":
    import other_mod


Cheers,


//arry/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/DMD2LIV6LRVTVOPXKS4BWCLVQTH2NB3O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-18 Thread Larry Hastings 


On 1/18/21 12:16 PM, Guido van Rossum wrote:
I do worry about the best practice getting worse if your PEP 649 is 
accepted.



A good part of what motivated me to start this second thread ("Let's Fix 
...") was how much worse best practice would become if PEP 649 is 
accepted.  But if we accept PEP 649, /and/ take steps to fix the 
semantics of annotations, I think the resulting best practice will be 
excellent in the long-run.


Let's assume for a minute that PEP 649 is accepted more-or-less like it 
is now.  (The name resolution approach is clearly going to change but 
that won't affect the discussion here.)  And let's assume that we also 
change the semantics so annotations are always defined (you can't delete 
them) and they're guaranteed to be either a dict or None.  (Permitting 
__annotations__ to be None isn't settled yet, but it's most similar to 
current semantics, so let's just assume it for now.)


Because the current semantics are kind of a mess, most people who 
examine annotations already have a function that gets the annotations 
for them.  Given that, I really do think the best approach is to gate 
the code on version 3.10, like I've described before:


   if python version >= 3.10:
    def get_annotations(o):
    return o.__annotations__
   else:
    def get_annotations(o):
    if isinstance(o, (type, types.ModuleType)):
    return o.__dict__.get("__annotations__", None)
    else:
    return o.__annotations__

This assumes returning None is fine.  If it had to always return a valid 
dict, I'd add "or {}" to the end of every return statement.


Given that it already has to be a function, I think this approach is 
readable and performant.  And, at some future time when the code can 
drop support for Python < 3.10, we can throw away the if statement and 
the whole else block, keeping just the one-line function.  At which 
point maybe we'd refactor away the function and just use 
"o.__annotations__" everywhere.



I concede that, in the short term, now we've got nine lines and two if 
statements to do something that /should/ be relatively 
straightforward--accessing the annotations on an object.  But that's 
where we find ourselves.  Current best practice is kind of a mess, and 
unfortunately PEP 649 breaks current best practice anyway.  My goal is 
to fix the semantics so that long-term best practice is sensible, easy, 
and obvious.



Cheers,


//arry/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/RHKSZ7RW2UTKV3MSION5SEPNV5WPHJFJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-18 Thread Guido van Rossum 
Hm. It's unfortunate that this would break code using what is *currently*
the best practice.

The saving grace seems that for *many* use cases the best practice is to
call typing.get_type_hints(). This is particularly useful for classes
because it includes annotations from base classes.

Also, for functions and modules I would recommend `getattr(o,
"__annotations__", None)` (perhaps with `or {}` added).

I would also honestly discount what dataclasses.py and typing.py have to
do. But what do 3rd party packages do when they don't want to use
get_type_hints() and they want to get it right for classes? That would give
an indication of how serious we should take breaking current best practice.

On Mon, Jan 18, 2021 at 1:10 PM Larry Hastings  wrote:

>
> On 1/18/21 12:16 PM, Guido van Rossum wrote:
>
> I do worry about the best practice getting worse if your PEP 649 is
> accepted.
>
>
> A good part of what motivated me to start this second thread ("Let's Fix
> ...") was how much worse best practice would become if PEP 649 is
> accepted.  But if we accept PEP 649, *and* take steps to fix the
> semantics of annotations, I think the resulting best practice will be
> excellent in the long-run.
>
> Let's assume for a minute that PEP 649 is accepted more-or-less like it is
> now.  (The name resolution approach is clearly going to change but that
> won't affect the discussion here.)  And let's assume that we also change
> the semantics so annotations are always defined (you can't delete them) and
> they're guaranteed to be either a dict or None.  (Permitting
> __annotations__ to be None isn't settled yet, but it's most similar to
> current semantics, so let's just assume it for now.)
>
> Because the current semantics are kind of a mess, most people who examine
> annotations already have a function that gets the annotations for them.
> Given that, I really do think the best approach is to gate the code on
> version 3.10, like I've described before:
>
> if python version >= 3.10:
> def get_annotations(o):
> return o.__annotations__
> else:
> def get_annotations(o):
> if isinstance(o, (type, types.ModuleType)):
> return o.__dict__.get("__annotations__", None)
> else:
> return o.__annotations__
>
> This assumes returning None is fine.  If it had to always return a valid
> dict, I'd add "or {}" to the end of every return statement.
>
> Given that it already has to be a function, I think this approach is
> readable and performant.  And, at some future time when the code can drop
> support for Python < 3.10, we can throw away the if statement and the whole
> else block, keeping just the one-line function.  At which point maybe we'd
> refactor away the function and just use "o.__annotations__" everywhere.
>
>
> I concede that, in the short term, now we've got nine lines and two if
> statements to do something that *should* be relatively
> straightforward--accessing the annotations on an object.  But that's where
> we find ourselves.  Current best practice is kind of a mess, and
> unfortunately PEP 649 breaks current best practice anyway.  My goal is to
> fix the semantics so that long-term best practice is sensible, easy, and
> obvious.
>
>
> Cheers,
>
>
> */arry*
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/Y7KUIKUHFQJ564HM2JNFFZF5EJHLNR2P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-18 Thread Inada Naoki 
On Tue, Jan 19, 2021 at 6:02 AM Larry Hastings  wrote:
>
>
> Oh, okay.  I haven't used the static type checkers, so it's not clear to me 
> what powers they do and don't have.  It was only a minor suggestion anyway.  
> Perhaps PEP 649 will be slightly inconvenient to people exploring their code 
> inside IPython.
>

Not only IPython, but many REPLs. Especially, Jupyter notebook is the
same to IPython.
We can see string annotations even in CPython REPL via pydoc.

```
>>> def func(a: "Optional[int]") -> "Optional[str]":
... ...
...
>>> help(func)

func(a: 'Optional[int]') -> 'Optional[str]'
```

Since this signature with type hints came from
inspect.signature(func), all tools using inspect.signature() will be
affected too.
I think Sphinx autodoc will be affected, but I am not sure.


> Or maybe it'd work if they gated the if statement on running in ipython?
>
> if typing.TYPE_CHECKING or os.path.split(sys.argv[0])[1] == "ipython3":
> import other_mod
>

It is possible for heavy modules, but not possible to avoid circular imports.
Additionally, there are some cases modules are not runtime importable.

* Optional dependency, user may not install it.
* Dummy modules having only "pyi" files.

If PEP 563 becomes the default, we can provide a faster way to get the
text signature without eval() annotated string. So eval() performance
is not a problem here.
Many type hinting use cases don't need type objects in runtime.
So I think PEP 563 is better for type hinting user experience.

Regards,

-- 
Inada Naoki  
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/QR2KOGAXR5T4GTLGL5NLPWSVWPGVFQAI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-18 Thread Larry Hastings 


On 1/18/21 3:42 PM, Inada Naoki wrote:

Many type hinting use cases don't need type objects in runtime.
So I think PEP 563 is better for type hinting user experience.



You mean, in situations where the user doesn't want to import the types, 
because of heavyweight imports or circular imports?  I didn't think 
those were very common.



//arry/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/P5MG5H5DFRG5UXOP7DSVZMLPAXEZTMZ6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-18 Thread Inada Naoki 
On Tue, Jan 19, 2021 at 8:54 AM Larry Hastings  wrote:
>
> On 1/18/21 3:42 PM, Inada Naoki wrote:
>
> Many type hinting use cases don't need type objects in runtime.
> So I think PEP 563 is better for type hinting user experience.
>
> You mean, in situations where the user doesn't want to import the types, 
> because of heavyweight imports or circular imports?  I didn't think those 
> were very common.
>

Personally, I dislike any runtime overhead caused by type hints. That
is one reason I don't use type hinting much for now.
I don't want to import modules used only in type hints. I don't want
to import even "typing".

I planned to use type hinting after I can drop Python 3.6 support and
use `from __future__ import annotations`.
And I love lightweight function annotation implementation (*) very much.

(*) https://github.com/python/cpython/pull/23316

I expect we can start to write type hints even in stdlibs, because it
doesn't require extra imports and overhead become very cheap.
Maybe, I am a minority. But I dislike any runtime overhead and extra
dependencies.

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/WLGZULJRK7PLQ37HJDJZPIZL5SM3NGF2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-18 Thread Larry Hastings 


On 1/18/21 2:39 PM, Guido van Rossum wrote:
Hm. It's unfortunate that this would break code using what is 
*currently* the best practice.


I can't figure out how to avoid it.  The problem is, current best 
practice sidesteps the class and goes straight to the dict.  How do we 
intercept that and run the code to lazy-calculate the annotations?


I mean, let's consider something crazy.  What if we change cls.__dict__ 
from a normal dict to a special dict that handles the __co_annotations__ 
machinery?  That might work, except, we literally allow users to supply 
their own cls.__dict__ via __prepare__.  So we can't rely on our special 
dict.


What if we change cls.__dict__ to a getset?  The user is allowed to set 
cls.__dict__, but when you get __dict__, we wrap the actual internal 
dict object with a special object that intercepts accesses to 
__annotations__ and handles the __co_annotations__ mechanism.  That 
might work but it's really crazy and unfortunate.  And it's remotely 
possible that a user might override __dict__ as a property, in a way 
that breaks this mechanism too.  So it's not guaranteed to always work.


I'm not suggesting we should do these things, I'm just trying to 
illustrate how hard I think the problem is.  If someone has a good idea 
how we can add the __co_annotations__ machinery without breaking current 
best practice I'd love to hear it.



Also, for functions and modules I would recommend `getattr(o, 
"__annotations__", None)` (perhaps with `or {}` added).


For functions you don't need to bother; fn.__annotations__ is guaranteed 
to always be set, and be either a dict or None. (Python will only ever 
set it to a dict, but the user is permitted to set it to None.)


I agree with your suggested best practice for modules as it stands today.

And actually, let me walk back something I've said before.  I believe 
I've said several times that "people treat classes and modules the 
same".  Actually that's wrong.


 * Lib/typing.py treats functions and modules the same; it uses
   getattr(o, '__annotations__', None).  It treats classes separately
   and uses cls.__dict__.get('__annotations__', {}).
 * Lib/dataclasses.py uses fn.__annotations__ for functions and
   cls.__dict__.get('__annotations__', {}) for classes.  It doesn't
   handle modules at all.
 * Lib/inspect.py calls Lib/typing.py to get annotations.  Which in
   retrospect I think is a bug, because annotations and type hints
   aren't the same thing.  (typing.get_type_hints changes None to
   type(None), it evaluates strings, etc).

So, for what it's worth, I literally have zero examples of people 
treating classes and modules the same when it comes to annotations.  
Sorry for the confusion!



I would also honestly discount what dataclasses.py and typing.py have 
to do. But what do 3rd party packages do when they don't want to use 
get_type_hints() and they want to get it right for classes? That would 
give an indication of how serious we should take breaking current best 
practice.


I'm not sure how to figure that out.  Off the top of my head, the only 
current third-party packages I can think of that uses annotations are 
mypy and attrs.  I took a quick look at mypy but I can't figure out what 
it's doing.


attrs does something a little kooky.  It access __annotations__ using a 
function called _has_own_attributes(), which detects whether or not the 
object is inheriting an attribute.  But it doesn't peek in __dict__, 
instead it walks the mro and sees if any of its base classes have the 
same (non-False) value for that attribute.


   
https://github.com/python-attrs/attrs/blob/a025629e36440dcc27aee0ee5b04d6523bcc9931/src/attr/_make.py#L343

Happily, that seems like it would continue to work even if PEP 649 is 
accepted.  That's good news!



Cheers,


//arry/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/REDSOWGE6CBW5INHFKC2MSPXIUN6V7KQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-18 Thread Guido van Rossum 
On Mon, Jan 18, 2021 at 4:34 PM Larry Hastings  wrote:

>
> On 1/18/21 2:39 PM, Guido van Rossum wrote:
>
> Hm. It's unfortunate that this would break code using what is *currently*
> the best practice.
>
> I can't figure out how to avoid it.  The problem is, current best practice
> sidesteps the class and goes straight to the dict.  How do we intercept
> that and run the code to lazy-calculate the annotations?
>
> I mean, let's consider something crazy.  What if we change cls.__dict__
> from a normal dict to a special dict that handles the __co_annotations__
> machinery?  That might work, except, we literally allow users to supply
> their own cls.__dict__ via __prepare__.  So we can't rely on our special
> dict.
>

There's a secret though. `cls.__dict__` is not actually a dict -- is a
mappingproxy. The proxy exists because we want to be able to intercept
changes to class attributes such as `__add__` or `__getattribute__` in
order to manipulate the C-level wrappers that implement such overloads.

So *perhaps* we could expand the mappingproxy class to trap read access to
`__annotations__` as a key to do your bidding. (The trick might be exposed
by things like .keys()  but that doesn't bother me as much.)

I honestly don't know how the mappingproxy and `__prepare__` interact. I
have to admit I've never used the latter. Presumably the mappingproxy still
plays a role because we'd still want to intercept e.g. `cls.__add__ = `.


> What if we change cls.__dict__ to a getset?  The user is allowed to set
> cls.__dict__, but when you get __dict__, we wrap the actual internal dict
> object with a special object that intercepts accesses to __annotations__
> and handles the __co_annotations__ mechanism.  That might work but it's
> really crazy and unfortunate.  And it's remotely possible that a user might
> override __dict__ as a property, in a way that breaks this mechanism too.
> So it's not guaranteed to always work.
>

Maybe such guarantees are overrated; in any case it looks like a rare
second-order effect (and we're already talking about esoteric usage
patterns).

I'm not suggesting we should do these things, I'm just trying to illustrate
> how hard I think the problem is.  If someone has a good idea how we can add
> the __co_annotations__ machinery without breaking current best practice I'd
> love to hear it.
>
> Also, for functions and modules I would recommend `getattr(o,
> "__annotations__", None)` (perhaps with `or {}` added).
>
> For functions you don't need to bother; fn.__annotations__ is guaranteed
> to always be set, and be either a dict or None.  (Python will only ever set
> it to a dict, but the user is permitted to set it to None.)
>
> I agree with your suggested best practice for modules as it stands today.
>
> And actually, let me walk back something I've said before.  I believe I've
> said several times that "people treat classes and modules the same".
> Actually that's wrong.
>
>- Lib/typing.py treats functions and modules the same; it uses
>getattr(o, '__annotations__', None).  It treats classes separately and uses
>cls.__dict__.get('__annotations__', {}).
>- Lib/dataclasses.py uses fn.__annotations__ for functions and
>cls.__dict__.get('__annotations__', {}) for classes.  It doesn't handle
>modules at all.
>- Lib/inspect.py calls Lib/typing.py to get annotations.  Which in
>retrospect I think is a bug, because annotations and type hints aren't the
>same thing.  (typing.get_type_hints changes None to type(None), it
>evaluates strings, etc).
>
> So, for what it's worth, I literally have zero examples of people treating
> classes and modules the same when it comes to annotations.  Sorry for the
> confusion!
>

Yeah, that part felt fishy -- basically classes are the only complicated
case here, because in order to construct the full set of annotations you
must walk the MRO.

Honestly *if* you are walking the MRO anyways, it probably doesn't matter
much if you use cls.__dict__.get('__annotations__') or getattr(cls,
'__annotations__') -- you might see some duplicates but you should
generally end up with the same overall set of annotations (though
presumably one could construct a counter-example using multiple
inheritance).

>
> I would also honestly discount what dataclasses.py and typing.py have to
> do. But what do 3rd party packages do when they don't want to use
> get_type_hints() and they want to get it right for classes? That would give
> an indication of how serious we should take breaking current best practice.
>
> I'm not sure how to figure that out.  Off the top of my head, the only
> current third-party packages I can think of that uses annotations are mypy
> and attrs.  I took a quick look at mypy but I can't figure out what it's
> doing.
>

Mypy is irrelevant because it reads your source code -- it doesn't ever run
your code to inspect `__annotations__`.

attrs does something a little kooky.  It access __annotations__ using a
> function called _has

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-18 Thread Greg Ewing 

On 19/01/21 1:13 pm, Inada Naoki wrote:

I don't want to import modules used only in type hints. I don't want
to import even "typing".


How about having a pseudo-module called __typing__ that is
ignored by the compiler:

from __typing__ import ...

would be compiled to a no-op, but recognised by type checkers.
If you want to do run-time typing stuff, you would use

from typing import ...

--
Greg
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/24MR5KVWP7LR3PCE7V44OTLJNXDLLNZX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-18 Thread Ethan Furman 

On 1/18/21 5:33 PM, Guido van Rossum wrote:

There's a secret though. `cls.__dict__` is not actually a dict -- is a mappingproxy. The proxy exists because we want to 
be able to intercept changes to class attributes such as `__add__` or `__getattribute__` in order to manipulate the 
C-level wrappers that implement such overloads.


So *perhaps* we could expand the mappingproxy class to trap read access to `__annotations__` as a key to do your 
bidding. (The trick might be exposed by things like .keys()  but that doesn't bother me as much.)


I honestly don't know how the mappingproxy and `__prepare__` interact.


`__prepare__` returns a dict-like namespace that is used as is.  `EnumMeta` uses `__prepare__` to return an instance of 
`_EnumDict`.


When `type.__new__` is called, whatever the namespace used to be is then converted into a normal Python dict, and a 
mappingproxy is returned for all further `cls.__dict__` requests.


--
~Ethan~
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/4RO6ZBRBPXCR6R5OTYCJXCO7PZWDGNEK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should python installation modify /etc/paths.d on Mac OS X?

2021-01-18 Thread Pankaj Jangid 
Ned Deily  writes:

> On Jan 14, 2021, at 09:22, Peng Yu  wrote:
>> The /etc/paths.d directory on Mac OS X contains the following
>> directory. Some packages have their PATH variables set there.
>> 
>> For Python, I have to set the PATH manually in my ~/.bashrc. Should
>> the Python developers consider setting PATH in /etc/paths.d as well?
>> 
>> ==> /etc/paths.d/100-rvictl <==
>> /Library/Apple/usr/bin
>> 
>> ==> /etc/paths.d/40-XQuartz <==
>> /opt/X11/bin
>> 
>> ==> /etc/paths.d/com.vmware.fusion.public <==
>> /Applications/VMware Fusion.app/Contents/Public
>
> Thanks for the suggestion. At first glance, that would seem to be an
> attractive option; it's something we've looked at
> recently. Unfortunately, there is a major usability issue with it on
> current macOS (10.15 Catalina and 11 Big Sur) where Apple now provides
> both a "python3' and a "pip3" stub in /usr/bin that link to versions
> they supply in current Xcode or Command Line Tools releases.  Those
> versions are supplied primarily to support other Xcode tools and are
> not intended to provide a full user Python installation.
>
> The problem is that paths added via /etc/paths.d are appended to PATH
> *after* /usr/local/bin and /usr/bin so it is not possible to override
> "python3" and "pip3" this way.
>
> We are still considering ways to improve the path management for
> python.org installers on macOS but simply relying on /etc/paths.d
> isn't sufficient, alas.

Probably very recently (Big Sur), but MacOS has stopped adding
/etc/paths.d into default PATH env variable. /etc/paths.d is loaded only
when you call

eval "$(/usr/libexec/path_helper)"

So, I had to add the above command in my ~/.bash_profile (interactive
login) to load my various utilities like TeX.

-- 
Regards,
Pankaj Jangid

GnuPG Fingerprint: 0B62 7424 3B26 A911 052A DDE6 7C95 6E6F F858 7689
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/SFNA6D6FQQARQHYRHGITOIA3SNCH7DZZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-18 Thread Ethan Furman 

On 1/18/21 5:53 PM, Ethan Furman wrote:

`__prepare__` returns a dict-like namespace that is used as is.  `EnumMeta` uses `__prepare__` to return an instance of 
`_EnumDict`.


When `type.__new__` is called, whatever the namespace used to be is then converted into a normal Python dict, and a 
mappingproxy is returned for all further `cls.__dict__` requests.


To be more precise, when `__prepare__` is called, there is no class, and 
therefore no `class.__dict__`.

--
~Ethan~
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/ABLK4FKI357SPCSKCYNG4BDLKD5UJHPU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-18 Thread Paul Sokolovsky 
Hello,

On Tue, 19 Jan 2021 14:31:49 +1300
Greg Ewing  wrote:

> On 19/01/21 1:13 pm, Inada Naoki wrote:
> > I don't want to import modules used only in type hints. I don't want
> > to import even "typing".  
> 
> How about having a pseudo-module called __typing__ that is
> ignored by the compiler:
> 
> from __typing__ import ...
> 
> would be compiled to a no-op, but recognised by type checkers.
> If you want to do run-time typing stuff, you would use

Please don't limit it to just "typing". There's a need for a module
which would handle "language-level" features, to not put newly added
things in the global namespace. By analogy with __future__, such a
module could be named __present__. Alternative names would be "lang" or
"python".

But analogy with __future__ is helpful, as there should be a place for
"pragma imports", which would change behavior of the programs, like
imports from __future__ do, except that features in __future__ are
destined to be "switchable" only temporary and become default later.
Breaking backward compatibility with each version has already become a
norm, but going further, even more radical changes would be required,
and so it should be possible to either enable or disable them, as part
of the standard, not temporary, language semantics, hence the idea of
__present__ as alternative to __future__.

> 
> from typing import ...
> 
> -- 
> Greg

[]

-- 
Best regards,
 Paul  mailto:[email protected]
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/LX7EB4CT5PCGNAE64RWQFI2OP63FYGR6/
Code of Conduct: http://python.org/psf/codeofconduct/