[Python-Dev] Re: Explicit markers for special C-API situations

2021-12-10 Thread Christian Heimes 

On 10/12/2021 03.08, Jim J. Jewett wrote:

Christian Heimes wrote:

On 09/12/2021 19.26, Petr Viktorin wrote:



If the code is the authoritative source of truth, we need a proper
parser to extract the information.  ... unfortunately I don't trust it
enough to let it define the API. Bugs in the parser could result in
the API definition silently changing.



There are other options than writing a new parser. GCC and Clang are
flexible. For example GCC can be extended with plugins and custom
attributes.


But they have the same problem ... it can be difficult to know if there is a 
subtle bug in someone's understanding of how the plugin interacts with, for 
example, nested ifndef.

The failure mode for an explicitly manually maintained text file is that 
something doesn't get added when it should, and the more conservative API 
consumers wait an extra release before using it.



Macros and ifndefs are not a problem. A GCC plugin for user-defined 
attributes hooks into the build process at a late stage. By the time the 
plugin hook is invoked, the precompiler has resolved all macros and 
ifdefs, and the C code has been parsed. The plugin operates on the same 
intermediate code as the compiler.



The approach would allow us to make the headers the authoritative source 
for most API and ABI symbols. I don't think that we can use it for 
macros. We can even include additional metadata in the custom attribute, 
e.g. version added


  PyAPI_ABI_FUNC("3.2", PyObject *) PyLong_FromLong(long);


We can convert Misc/stable_abi.txt into an auto-generated file. The file 
should still stay in git, so we can use it to verify the stable ABI in CI.


Christian
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/5GSBMLBCXC3THYZYXC5E627CGQELE6JI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Explicit markers for special C-API situations

2021-12-10 Thread Petr Viktorin 

On 10. 12. 21 11:55, Christian Heimes wrote:

On 10/12/2021 03.08, Jim J. Jewett wrote:

Christian Heimes wrote:

On 09/12/2021 19.26, Petr Viktorin wrote:



If the code is the authoritative source of truth, we need a proper
parser to extract the information.  ... unfortunately I don't trust it
enough to let it define the API. Bugs in the parser could result in
the API definition silently changing.



There are other options than writing a new parser. GCC and Clang are
flexible. For example GCC can be extended with plugins and custom
attributes.


But they have the same problem ... it can be difficult to know if 
there is a subtle bug in someone's understanding of how the plugin 
interacts with, for example, nested ifndef.


The failure mode for an explicitly manually maintained text file is 
that something doesn't get added when it should, and the more 
conservative API consumers wait an extra release before using it.



Macros and ifndefs are not a problem.


They are: we want to find PyErr_SetExcFromWindowsErr on all systems, and 
include it in the docs.



A GCC plugin for user-defined 
attributes hooks into the build process at a late stage. By the time the 
plugin hook is invoked, the precompiler has resolved all macros and 
ifdefs, and the C code has been parsed. The plugin operates on the same 
intermediate code as the compiler.



The approach would allow us to make the headers the authoritative source 
for most API and ABI symbols. I don't think that we can use it for 
macros. We can even include additional metadata in the custom attribute, 
e.g. version added


   PyAPI_ABI_FUNC("3.2", PyObject *) PyLong_FromLong(long);


This looks a bit awkward already, and if/when we start including e.g. 
"version removed" for PyAPI_ABI_ONLY, it'll get worse.



We can convert Misc/stable_abi.txt into an auto-generated file. The file 
should still stay in git, so we can use it to verify the stable ABI in CI.


We can, but genuinely I think it works better as a source of truth than 
a generated artifact. Changes to it should be deliberate.
I get that not everyone will agree with that. But it's also *much* 
easier to maintain the current "best-effort" checks (which can punt on a 
few edge cases) than add an all-encompassing, tested parser-based generator.


Not everything needs to be automated :)



Eric said:

The tooling is a secondary concern to my point.  Mostly, I wish the 
declarations in the header files had the extra classifications, rather than 
having to remember to refer to a separate text file.


This part sounds like a good idea.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/HSIUJSE6QW2U4QEEHFOZSQIDD7Q3DT7G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should isinstance call __getattribute__?

2021-12-10 Thread Steven D'Aprano 
On Thu, Dec 09, 2021 at 05:19:00PM +0100, Ronald Oussoren wrote:

> https://mail.python.org/pipermail/python-dev/2015-October/141953.html 
> is an old thread about the difference between type(x)/Py_TYPE(x) and 
> x.__class__ that contains some insight about this.

Thanks for the link Ronald, I remember that thread. It didn't really 
clarify things to me at the time, and re-reading it, it still doesn't.

> Proxy types are one use case, although with some sharp edges.

I'm not looking for use cases. I'm looking for a better understanding of 
how type() and isinstance() (and presumably issubclass) work. The best I 
can see is that type() sometimes believes __class__ but not always, that 
you can sometimes change __class__ but not always, but the rules that 
control when and why (or why not) are not clear or documented, as far 
as I can see.

Is there a reference for how type(obj) and isinstance(obj, T) are 
intended to work, or is the implementation the only reference?

Thanks in advance,


-- 
Steve
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/GB2S2SMNDGS5UV5GG6O7HQUQSZP27OOI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should isinstance call __getattribute__?

2021-12-10 Thread Ronald Oussoren via Python-Dev 


> On 10 Dec 2021, at 14:40, Steven D'Aprano  wrote:
> 
> On Thu, Dec 09, 2021 at 05:19:00PM +0100, Ronald Oussoren wrote:
> 
>> https://mail.python.org/pipermail/python-dev/2015-October/141953.html 
>> is an old thread about the difference between type(x)/Py_TYPE(x) and 
>> x.__class__ that contains some insight about this.
> 
> Thanks for the link Ronald, I remember that thread. It didn't really 
> clarify things to me at the time, and re-reading it, it still doesn't.

That’s because the difference between the two is not clear, and it doesn’t help 
that C extensions (and CPython itself) use an API that always look at the 
object’s type slot and not at the ``__class__`` attribute (with APIs such as 
``PyObject_TypeCheck`` and ``PyDict_Check``). 

> 
>> Proxy types are one use case, although with some sharp edges.
> 
> I'm not looking for use cases. I'm looking for a better understanding of 
> how type() and isinstance() (and presumably issubclass) work. The best I 
> can see is that type() sometimes believes __class__ but not always, that 
> you can sometimes change __class__ but not always, but the rules that 
> control when and why (or why not) are not clear or documented, as far 
> as I can see.
> 
> Is there a reference for how type(obj) and isinstance(obj, T) are 
> intended to work, or is the implementation the only reference?

I’m not sure how much of this is documented to be honest. If the documentation 
is lacking there’s a change for someone to dig deep and help writing the 
documentation ;-)

Changing the type of an instance by assigning to ``__class__`` is basically 
allowed when the C layout for instances of the two classes are compatible, the 
implementation contains the details about this.  IIRC this requires that both 
classes inherit from a shared base class and none of the intermediate classes 
introduce new slots (or the C equivalent of this). 

Ronald

> 
> Thanks in advance,
> 
> 
> -- 
> Steve
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/[email protected]/message/GB2S2SMNDGS5UV5GG6O7HQUQSZP27OOI/
> Code of Conduct: http://python.org/psf/codeofconduct/

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/MIBOFO7CZM3TBKA5C3IRIHGB62MPCZFS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Summary of Python tracker Issues

2021-12-10 Thread Python tracker 

ACTIVITY SUMMARY (2021-12-03 - 2021-12-10)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7195 (-57)
  closed 50572 (+123)
  total  57767 (+66)

Open issues with patches: 2859 


Issues opened (44)
==

#23469: Delete Misc/*.wpr files
https://bugs.python.org/issue23469  reopened by berker.peksag

#28533: Remove asyncore, asynchat and smtpd modules
https://bugs.python.org/issue28533  reopened by vstinner

#31184: Fix data descriptor detection in inspect.getattr_static
https://bugs.python.org/issue31184  reopened by davidhalter

#45620: A misleading url in 'Floating Point Arithmetic' page
https://bugs.python.org/issue45620  reopened by eric.smith

#45798: Move _decimal build setup into configure
https://bugs.python.org/issue45798  reopened by ned.deily

#45975: Simplify some while-loops with walrus operator
https://bugs.python.org/issue45975  opened by nickdrozd

#45977: Unexpected effect of sys.pycache_prefix = ""
https://bugs.python.org/issue45977  opened by andrei.avk

#45978: deepfreeze opaquely fails on Windows when building from Visual
https://bugs.python.org/issue45978  opened by Dennis Sweeney

#45979: Fix Tkinter tests with old Tk
https://bugs.python.org/issue45979  opened by serhiy.storchaka

#45981: Get raw file name in bytes from ZipFile
https://bugs.python.org/issue45981  opened by accelerator0099

#45985: AttributeError from @property inadvertantly flows into __getat
https://bugs.python.org/issue45985  opened by koreno

#45988: inspect.signature fails on a @staticmethod
https://bugs.python.org/issue45988  opened by PhilipVinc

#45990: Exception notes need more documentation
https://bugs.python.org/issue45990  opened by cool-RR

#45991: Improve ambiguous docstrings in pkgutil
https://bugs.python.org/issue45991  opened by khock

#45992: distutils paths are scattered between PythonXY and PythonXY-32
https://bugs.python.org/issue45992  opened by uranusjr

#45994: Add simple usage to email module
https://bugs.python.org/issue45994  opened by tarao1006

#45995: string formatting: normalize negative zero
https://bugs.python.org/issue45995  opened by John Belmonte

#45996: Worse error from asynccontextmanager in Python 3.10
https://bugs.python.org/issue45996  opened by Dima.Tisnek

#45997: asyncio.Semaphore waiters deque doesn't work
https://bugs.python.org/issue45997  opened by hyzyla

#45998: Document best practice for including and linking python framew
https://bugs.python.org/issue45998  opened by ronaldoussoren

#46005: [doc] replace 'distutils' examples with 'setuptools'
https://bugs.python.org/issue46005  opened by elmjag

#46006: [subinterpreter] _PyUnicode_EqualToASCIIId() issue with subint
https://bugs.python.org/issue46006  opened by vstinner

#46010: Cookie cutter templates to allow variable fields and data tran
https://bugs.python.org/issue46010  opened by Francisco

#46011: Python 3.10 email returns invalid Date: header unchanged.
https://bugs.python.org/issue46011  opened by msapiro

#46013: Confusing period in object.__hash__ doc
https://bugs.python.org/issue46013  opened by JMcB17

#46014: functools.singledispatch does not support Union types
https://bugs.python.org/issue46014  opened by evansd

#46017: Tutorial incorrectly refers to skits rather than sketches.
https://bugs.python.org/issue46017  opened by k2k

#46020: Optimize long_pow for the common case
https://bugs.python.org/issue46020  opened by rhettinger

#46021: fcntl module update supports FreeBSD F_KINFO flag
https://bugs.python.org/issue46021  opened by devnexen

#46022: Multiprocessing.Server.serve_forever runs sys.exit()
https://bugs.python.org/issue46022  opened by bar.harel

#46023: Modules/makesetup generated rules ignore *disabled*
https://bugs.python.org/issue46023  opened by christian.heimes

#46024: Different behaviour with zipfile
https://bugs.python.org/issue46024  opened by flvn.dev

#46026: importlib.resources.read_text() raises FileNotFound
https://bugs.python.org/issue46026  opened by Zac Hatfield-Dodds

#46027: email.utils.parsedate_to_datetime() handling of - offset
https://bugs.python.org/issue46027  opened by fdrake

#46028: 3.11.0a3: under tox, sys._base_executable is wrong
https://bugs.python.org/issue46028  opened by nedbat

#46030: socket module add couple of FreeBSD constants
https://bugs.python.org/issue46030  opened by dcarlier

#46031: add POP_JUMP_IF_NOT_NONE and POP_JUMP_IF_NONE
https://bugs.python.org/issue46031  opened by penguin_wwy

#46032: functools' singledispatch does not support GenericAlias
https://bugs.python.org/issue46032  opened by kumaraditya303

#46033: Duplicated sentence in for statement documentation
https://bugs.python.org/issue46033  opened by michcio1234

#46035: mimetypes.guess_type returns deprecated mimetype application/x
https://bugs.python.org/issue46035  opened by milahu

#46036: Single-phase initialized modules

[Python-Dev] Sad news from Zurich

2021-12-10 Thread Guido van Rossum 
A former core dev who works at Google just passed the news that Fredrik
Lundh (also known as Effbot) has died.

Fredrik was an early Python contributor (e.g. Elementtree and the 're'
module) and his enthusiasm for the language and community were inspiring
for all who encountered him or his work. He spent countless hours on
comp.lang.python answering questions from newbies and advanced users alike.

He also co-founded an early Python startup, Secret Labs AB, which among
other software released an IDE named PythonWorks. Fredrik also created the
Python Imaging Library (PIL) which is still THE way to interact with images
in Python, now most often through its Pillow fork. His effbot.org site was
a valuable resource for generations of Python users, especially its Tkinter
documentation.

Fredrik's private Facebook page contains the following message from
November 25 by Ylva Larensson (translated from Swedish):

"""

It is with such sorrow and pain that I write this. Fredrik has left us
suddenly.

"""

A core dev wrote: "I felt privileged to be able to study Fredrik's code and
to read his writing. He was a huge asset to our community in the early
days. I enjoyed his sense of humor as well. I'm sad that he passed away."

We will miss him.



-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/36Q5QBILL3QIFIA3KHNGFBNJQKXKN7SD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Sad news from Zurich

2021-12-10 Thread Hasan Diwan 
My condolences to all who knew Fredrik. -- H
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/3SWFS77UCVDKMJOBCDGDTVTRDM3VFMV2/
Code of Conduct: http://python.org/psf/codeofconduct/