[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-03-31 Thread Victor Stinner 
On Wed, Mar 30, 2022 at 9:26 PM Sebastian Berg
 wrote:
> That is fair, although there are documented underscored ones:
> https://docs.python.org/3/search.html?q=_Py
>
> I suppose that means all bets are off _unless_ it is documented or
> later adopted as stable API (e.g. `_PyObject_Vectorcall`).
>
> With that, the only "not obviously OK" use in NumPy that I am aware of
> is `_Py_HashDouble` (it seems undocumented).
>
> Maybe "unless documented" is just a clear enough distinction in
> practice.
> Although, to some degree, I think it would be clearer if symbols that
> have a realistic chance of changing in bug-fix releases had an
> additional safe-guard.

Since Python 3.7, there is a work-in-progress to (1) better hide
internal C API functions and to (2) promote *private* C API functions
being used by 3rd party projects as documented and well tested
*public* functions.

An example of (2) is the addition of float pack/unpack public
functions, like PyFloat_Pack8() and PyFloat_Unpack8():
https://docs.python.org/dev/c-api/float.html#pack-and-unpack-functions

There were previously known as private _PyFloat_Pack8() and
_PyFloat_Unpack8() functions. They are used by a few serialization
projects like msgpack. When they were made public, tests were added
and the existing comments were converted to documentation and
enhanced.

I discovered that these private functions were used when I moved them
to the internal C API in bpo-46906 (1) and it broke a few projects.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/DT2PUGWPOSZOJZLR4VOMPS6QOS3PDRYR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

2022-03-31 Thread Ronald Oussoren via Python-Dev 


> On 29 Mar 2022, at 19:51, Brett Cannon  wrote:
> 
> 
> 
> On Tue, Mar 29, 2022 at 8:58 AM Ronald Oussoren  > wrote:
> 
> 
>> On 29 Mar 2022, at 00:34, Brett Cannon > > wrote:
>> 
>> 
>> 
>> Once 
>> https://mail.python.org/archives/list/[email protected]/thread/5EUZLT5PNA4HT42NGB5WVN5YWW5ASTT5/
>>  
>> 
>>  is considered resolved, the next part of my "what is the stdlib" plan is to 
>> finally try to suss all of this out and more-or-less write a stdlib policy 
>> PEP so we stop talking about this. My guess it will be more of guidance 
>> about what we want the stdlib to be and thus guide what things belong in it. 
>> No ETA on that work since I also have four other big Python projects on the 
>> go right now whose work I am constantly alternating between.
> 
> Having such a policy is a good thing and helps in evolving the stdlib, but I 
> wonder if the lack of such a document is the real problem.   IMHO the main 
> problem is that the CPython team is very small and therefore has little 
> bandwidth for maintaining, let alone evolving, large parts of the stdlib.  In 
> that it doesn’t help that some parts of the stdlib have APIs that make it 
> hard to make modifications (such as distutils where effectively everything is 
> part of the public API).  Shrinking the stdlib helps in the maintenance 
> burden, but feels as a partial solution. 
> 
> You're right that is the fundamental problem. But for me this somewhat stems 
> from the fact that we don't have a shared understanding of what the stdlib 
> is,  and so the stdlib is a bit unbounded in its size and scope. That leads 
> to a stdlib which is hard to maintain.

That (the hard to maintain part) is not necessarily true, if we had enough 
resources…. In theory the stdlib could be split into logical parts with teams 
that feel responsible for those parts (similar to having maintainers sign up 
for various platforms).  That doesn’t work because of the small team, and 
partially also due to necessarily having very strict rules w.r.t. backward 
compatibility. 


> It's just like dealing with any scarce resource: you try to cut back on your 
> overall use as best as you can and then become more efficient with what you 
> must still consume; I personally think we don't have an answer to the "must 
> consume" part of that sentence that leads us to "cut back" to a size we can 
> actually keep maintained so we don't have 1.6K open PRs 
> .

I agree, but this is something to state explicitly when describing what should 
and should not be in scope for the stdlib.   I’m a fan of a batteries included 
stdlib, but with our current resources we cannot afford to have some bits in 
the stdlib that would “obviously” be a candidate for a modern batteries 
included stdlib, such as a decent HTTP stack with support for HTTP/1, /2 and 
/3. 

Ronald

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/JS23QUVXW3HZHQFMZBOXUCKT332KGRYS/
Code of Conduct: http://python.org/psf/codeofconduct/