[Python-Dev] Regressions caused the recent work on ceval.c and frame objects

2021-09-19 Thread Victor Stinner 
Hi,

The recent optimization work on ceval.c and frame objects introduced
regressions and the situation is stuck for 4 months (bpo-43760).

Right now, maybe the best would be to revert the 2 commits in the 3.10
branch, to get more time in Python 3.11 development cycle to solve
these issues.


(1) "The DISPATCH() macro is not as efficient as it could be (move
PyThreadState.use_tracing)"
https://bugs.python.org/issue43760

This change introduced an incompatible C API change. It's not
documented in What's New in Python 3.10 and there is no solution for
the 4 broken projects (including Cython).

I proposed a C API but so far, nobody implemented it.

Another option is to fix each project since the list is short. Right
now, only 4 projects are known to be broken. Fixing Cython is not
enough, you need to get a new release of broken projects (using
Cython) to regenerate the C code with the updated Cython.


(2) "Performance regression 3.10b1 and later on Windows: Py_DECREF()
not inlined in PGO build"
https://bugs.python.org/issue45116

Changes made in bpo-43760 caused a performance regression on the PGO
build of Windows.

It's a tricky issue about thresholds in compiler PGO optimization,
inlining or not static inline functions, number of statements per
function, etc.

It was proposed to workaround the specific case of the huge
_PyEval_EvalFrameDefault() function (3500 lines of C code) by
converting again some static inline functions to macros. They were
macros in Python 3.8 and were fine in Python 3.9. The performance
regression with the Windows PGO build was introduced by the recent
ceval.c work.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/XLEMW7PWYGIK2RHHICO3CNITZ4ETO3OZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should the definition of an "(async) iterator" include __iter__?

2021-09-19 Thread Steve Holden 
I understood that _iterables_ are required to have an __iter__ method, not
iterators.

Therefore, are we simply discussing whether all iterators should be
iterable? At the moment the CPython implementation does't require that
AFAIK.

regards
 Steve

On Tue, Sep 14, 2021 at 8:39 PM Guido van Rossum  wrote:

> My view of this is:
>
> A. It's not an iterator if it doesn't define `__next__`.
>
> B. It is strongly recommended that iterators also define `__iter__`.
>
> In "standards" language, I think (A) is MUST and (B) is merely OUGHT or
> maybe SHOULD.
>
> On Tue, Sep 14, 2021 at 12:30 PM Brett Cannon  wrote:
>
>> Over in https://github.com/python/typeshed/issues/6030 I have managed to
>> kick up a discussion over what exactly an "iterator" is. If you look at
>> https://docs.python.org/3/library/functions.html#iter you will see the
>> docs say it "Return[s] an iterator
>>  object." Great,
>> but you go the glossary definition of "iterator" at
>> https://docs.python.org/3/glossary.html#term-iterator you will see it
>> says "[i]terators are required to have an __iter__()
>> 
>> method" which neither `for` nor `iter()` actually enforce.
>>
>> Is there something to do here? Do we loosen the definition of "iterator"
>> to say they *should* define __iter__? Leave it as-is with an
>> understanding that we know that it's technically inaccurate for iter() but
>> that we want to encourage people to define __iter__? I'm assuming people
>> don't want to change `for` and `iter()` to start requiring __iter__ be
>> defined if we decided to go down the "remove the __aiter__ requirement"
>> from aiter() last week.
>>
>> BTW all of this applies to async iterators as well.
>> ___
>> Python-Dev mailing list -- [email protected]
>> To unsubscribe send an email to [email protected]
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/[email protected]/message/3W7TDX5KNVQVGT5CUHBK33M7VNTP25DZ/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/OICGRBPLXO6WXO4CHTGUK46WIHO7PDUU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/KHDMNMW6XEPYOZ5AQ22AN6YTN2POMHQE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Regressions caused the recent work on ceval.c and frame objects

2021-09-19 Thread dw-git 
Are you sure Cython is still broken? It looks like it was fixed back in May 
(https://github.com/cython/cython/issues/4153) and all the tests look to be 
passing on the 3.10-dev CI run for Cython. I think it only affected the 
profiling feature on Cython (which most people will have turned off) so 
probably won't cause widespread breakage.

Which is not to say that you shouldn't fix the issue, but I don't think it's a 
disaster from Cython's point of view.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/B6OLXTTP5KQVNFLZD2WWV4YSYNSUZX6Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Regressions caused the recent work on ceval.c and frame objects

2021-09-19 Thread Miro Hrončok 

On 19. 09. 21 21:34, [email protected] wrote:

Are you sure Cython is still broken? It looks like it was fixed back in May 
(https://github.com/cython/cython/issues/4153) and all the tests look to be 
passing on the 3.10-dev CI run for Cython. I think it only affected the 
profiling feature on Cython (which most people will have turned off) so 
probably won't cause widespread breakage.

Which is not to say that you shouldn't fix the issue, but I don't think it's a 
disaster from Cython's point of view.


Cython is fixed. However all the projects that ship Cython pre-generated C 
sources in sdists on PyPI that are affected and were not re-generated will not 
compile on Python 3.10. (Which might already be the case for different changes 
that affected Cython.)


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/YHPXTAOHGYDKYF4WY42L2KSTDLR65Q2T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Regressions caused the recent work on ceval.c and frame objects

2021-09-19 Thread Thomas Grainger 
Are projects that ship pre-compiled wheels impacted? Eg twisted-iocpsupport
?

On Sun, 19 Sep 2021, 22:09 Miro Hrončok,  wrote:

> On 19. 09. 21 21:34, [email protected] wrote:
> > Are you sure Cython is still broken? It looks like it was fixed back in
> May (https://github.com/cython/cython/issues/4153) and all the tests look
> to be passing on the 3.10-dev CI run for Cython. I think it only affected
> the profiling feature on Cython (which most people will have turned off) so
> probably won't cause widespread breakage.
> >
> > Which is not to say that you shouldn't fix the issue, but I don't think
> it's a disaster from Cython's point of view.
>
> Cython is fixed. However all the projects that ship Cython pre-generated C
> sources in sdists on PyPI that are affected and were not re-generated will
> not
> compile on Python 3.10. (Which might already be the case for different
> changes
> that affected Cython.)
>
> --
> Miro Hrončok
> --
> Phone: +420777974800
> IRC: mhroncok
>
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/YHPXTAOHGYDKYF4WY42L2KSTDLR65Q2T/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/55PGGLILX3VCKTJELZKFOKEKUEGKNQIY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Regressions caused the recent work on ceval.c and frame objects

2021-09-19 Thread Miro Hrončok 

On 20. 09. 21 0:10, Thomas Grainger wrote:

Are projects that ship pre-compiled wheels impacted? Eg twisted-iocpsupport ?


I guess that if they managed to compile with 3.10, they shouldn't be.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/TDGG4VFRZO5CDKTUH4AXYDLVXOKNLZZE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Regressions caused the recent work on ceval.c and frame objects

2021-09-19 Thread Christopher Barker 
Will all packages that use Cython have to upgrade Cython to work with 3.10?

That can be a pretty heavy lift. A lot of us pin Cython.

-CHB


On Sun, Sep 19, 2021 at 4:32 PM Miro Hrončok  wrote:

> On 20. 09. 21 0:10, Thomas Grainger wrote:
> > Are projects that ship pre-compiled wheels impacted? Eg
> twisted-iocpsupport ?
>
> I guess that if they managed to compile with 3.10, they shouldn't be.
>
> --
> Miro Hrončok
> --
> Phone: +420777974800
> IRC: mhroncok
>
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/TDGG4VFRZO5CDKTUH4AXYDLVXOKNLZZE/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/NVVWCZBZLO2T26VQLGLH34TTOHCCHR7F/
Code of Conduct: http://python.org/psf/codeofconduct/