[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

2021-04-03 Thread Mark Shannon 

Hi Brandt,

On 02/04/2021 8:41 pm, Brandt Bucher wrote:

Mark Shannon wrote:

On 02/04/2021 7:19 am, Brandt Bucher wrote:

I agree that self-matching classes should absolutely allow keyword matches. I 
had no idea the PEP forbade it.

PEP 634 allows it.


PEP 634 says:


For a number of built-in types (specified below), a single positional 
subpattern is accepted which will match the entire subject; for these types no 
keyword patterns are accepted.


(https://www.python.org/dev/peps/pep-0634/#class-patterns)


I was relying on the "reference" implementation, which is also in the PEP.

>>> match 0:
... case int(imag=0):
...print ("Experimentally, int supports keyword matching.")
...
Experimentally, int supports keyword matching.

I take this as +1 for having more precisely defined semantics for 
pattern matching :)





Most checks are cheap though.
Checking for duplicates in `__match_args__` can be done at class creation time, 
and checking for duplicates in the pattern can be done at compile time.


I assume the compile-time check only works for named keyword attributes. The 
current implementation already does this.

-1 on checking `__match_args__` anywhere other than the match block itself.


I'm curious, why?
It is much faster *and* gives better error messages to check 
`__match_args__` at class creation time.




Guido van Rossum wrote:

On Fri, Apr 2, 2021 at 3:38 AM Mark Shannon [email protected] wrote:

Are there are any use-cases?
The test-case `int(real=0+0j, imag=0-0j)` is contrived, but I'm struggling to 
come up with less contrived examples for any of float, list, dict, tuple, str.

There could be a subclass that adds an attribute. That's still contrived  
though.


I could see the case for something like `case defaultdict({"Spam": s}, 
default_factory=f)`. I certainly don't think it should be forbidden.


It is forbidden in the PEP, as written, correct?
OOI, have you changed your mind, or was that an oversight in the original?

Cheers,
Mark.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/R7ZIAAQNL2CN3K3M3DMR2IQNC6DLD7FF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

2021-04-03 Thread Mark Shannon 

Hi Guido,

On 02/04/2021 10:05 pm, Guido van Rossum wrote:
On Fri, Apr 2, 2021 at 12:43 PM Brandt Bucher > wrote:


Mark Shannon wrote:
 > On 02/04/2021 7:19 am, Brandt Bucher wrote:
 > > I agree that self-matching classes should absolutely allow
keyword matches. I had no idea the PEP forbade it.
 > PEP 634 allows it.

PEP 634 says:

 > For a number of built-in types (specified below), a single
positional subpattern is accepted which will match the entire
subject; for these types no keyword patterns are accepted.

(https://www.python.org/dev/peps/pep-0634/#class-patterns
)


But that's not what the implementation does. It still supports keyword 
patterns for these types -- and (as I've said earlier in this thread) I 
think the implementation is correct.


 > Most checks are cheap though.
 > Checking for duplicates in `__match_args__` can be done at class
creation time, and checking for duplicates in the pattern can be
done at compile time.

I assume the compile-time check only works for named keyword
attributes. The current implementation already does this.

-1 on checking `__match_args__` anywhere other than the match block
itself.


Agreed.


Why? (I also asked Brandt this)

It is far more efficient to check `__match_args__` at class creation (or 
class attribute assignment) time.


The most efficient way to check in the match block is to check at class 
creation time anyway and store a flag whether  `__match_args__` is 
legal. In the match block we would check this flag, then proceed.


It seems silly to know that there will be a runtime error, but not act 
on that information, allowing latent bugs could have been reported.


Cheers,
Mark.



--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 



___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/S45E7DCDXU2LTBSNYYXSIDH6CL7CVDIK/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/EN25PHRQO55AORYWL5SB63JMLVFT6V2L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

2021-04-03 Thread Łukasz Langa 
The memory layout of PyThreadState was unintentionally changed in the recent 
3.9.3 bugfix release. This leads to crashes on 32-bit systems when importing 
binary extensions compiled for Python 3.9.0 - 3.9.2. This is a regression.

We will be releasing a hotfix 3.9.4 around 24 hours from now to address this 
issue and restore ABI compatibility with C extensions built for Python 3.9.0 - 
3.9.2.

Details:
https://bugs.python.org/issue43710

- Ł___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/OUCNKMFBNGFK2AI4B7S7MF5O6UVLBSMB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

2021-04-03 Thread Guido van Rossum 
On Sat, Apr 3, 2021 at 4:15 AM Mark Shannon  wrote:

> Hi Brandt,
>
> On 02/04/2021 8:41 pm, Brandt Bucher wrote:
> > Mark Shannon wrote:
> >> On 02/04/2021 7:19 am, Brandt Bucher wrote:
> >>> I agree that self-matching classes should absolutely allow keyword
> matches. I had no idea the PEP forbade it.
> >> PEP 634 allows it.
> >
> > PEP 634 says:
> >
> >> For a number of built-in types (specified below), a single positional
> subpattern is accepted which will match the entire subject; for these types
> no keyword patterns are accepted.
> >
> > (https://www.python.org/dev/peps/pep-0634/#class-patterns)
>
> I was relying on the "reference" implementation, which is also in the PEP.
>

But it's nor normative. However...


>  >>> match 0:
> ... case int(imag=0):
> ...print ("Experimentally, int supports keyword matching.")
> ...
> Experimentally, int supports keyword matching.
>

In this case I propose adjusting the PEP text. See
https://github.com/python/peps/pull/1908


> I take this as +1 for having more precisely defined semantics for
> pattern matching :)
>

Certainly I see it as +1 for having the semantics independently verified.

 [...]

> > Guido van Rossum wrote:
> >> On Fri, Apr 2, 2021 at 3:38 AM Mark Shannon [email protected] wrote:
> >>> Are there are any use-cases?
> >>> The test-case `int(real=0+0j, imag=0-0j)` is contrived, but I'm
> struggling to come up with less contrived examples for any of float, list,
> dict, tuple, str.
> >> There could be a subclass that adds an attribute. That's still
> contrived  though.
> >
> > I could see the case for something like `case defaultdict({"Spam": s},
> default_factory=f)`. I certainly don't think it should be forbidden.
>
> It is forbidden in the PEP, as written, correct?
> OOI, have you changed your mind, or was that an oversight in the original?
>

I was surprised to find this phrase in the PEP, so I suspect that it was
just a mistake when I wrote that section of the PEP. I can't find a similar
restriction in PEP 622 (the original pattern matching PEP).

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/6TML5LNQUVPMBWHN3FXOZP7J2GNVBBES/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

2021-04-03 Thread Guido van Rossum 
On Sat, Apr 3, 2021 at 4:20 AM Mark Shannon  wrote:

> Hi Guido,
>
> On 02/04/2021 10:05 pm, Guido van Rossum wrote:
> > On Fri, Apr 2, 2021 at 12:43 PM Brandt Bucher  > [...]
> > -1 on checking `__match_args__` anywhere other than the match block
> > itself.
> >
> > Agreed.
>
> Why? (I also asked Brandt this)
>
> It is far more efficient to check `__match_args__` at class creation (or
> class attribute assignment) time.
>

Okay, now we're talking. If you check it on both class definition and at
attribute assignment time I think that's fine (now that it's a tuple).

But I don't think the specification (in whatever PEP) needs to specify that
it *must* be checked at that time. So I think the current implementation is
fine as well (once we change it to accept only tuples).


> The most efficient way to check in the match block is to check at class
> creation time anyway and store a flag whether  `__match_args__` is
> legal. In the match block we would check this flag, then proceed.
>

Yeah, nice optimization.


> It seems silly to know that there will be a runtime error, but not act
> on that information, allowing latent bugs could have been reported.
>

Well, usually that is The Python Way. There are a lot of things that could
be detected statically quite easily (without building something like mypy)
but that aren't. Often that's due to historical accidents (in the past we
were even less able to do the simplest static checks), so it's fine to do
this your way.

BTW we previously discussed whether `__match_args__` can contain
duplicates. I thought the PEP didn't state either way, but I was wrong: it
explicitly disallows it, matching the implementation. PEP 634 says on line
503:
```
- For duplicate keywords, ``TypeError`` is raised.
```
Given that there is no inconsistency here, I am inclined to keep it that
way. If we find a better use case to allow duplicates we can always loosen
up the implementation; it's not so simple the other way around.

FWIW I am also submitting https://github.com/python/peps/pull/1909 to make
`__match_args__` a tuple only, which we all seems to agree on.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/TCRU227DPZ5YMDO7EO5RUV354VUAW4OY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

2021-04-03 Thread Miro Hrončok 

On 03. 04. 21 21:44, Łukasz Langa wrote:
The memory layout of PyThreadState was unintentionally changed in the recent 
3.9.3 bugfix release. This leads to crashes on 32-bit systems when importing 
binary extensions compiled for Python 3.9.0 - 3.9.2. This is a regression.


We will be releasing a hotfix 3.9.4 around 24 hours from now to address this 
issue and restore ABI compatibility with C extensions built for Python 3.9.0 - 
3.9.2.


Thanks for the hotifx.

However, I need to ask: Would this also happen if there was a rc version of 
3.9.3?

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/FNA54YSXUGMWM3ON4PSGFHHFYMAPFFD7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

2021-04-03 Thread Brandt Bucher 
Mark Shannon said:
> I was relying on the "reference" implementation, which is also in the PEP.

Can you please stop putting scare quotes around "reference implementation"? 
You've done it twice now, and it's been a weekend-ruiner for me each time.

I've put months of work into writing and improving CPython's current pattern 
matching implementation, mostly on nights and weekends. I don't know whether 
it's intentional or not, but when you say things like that it instantly 
devalues all of my hard work in front of everyone on the list.

For such a huge feature, I'm honestly quite amazed that this is the only issue 
we've found since it was merged over a month ago (and both authors have agreed 
that it needs to be fixed in the PEP, not the implementation). The PR 
introducing this behavior was reviewed by at least a half-dozen people, 
including you.

The last time you said something like this, I just muted the thread. Let's 
please keep this respectful; we're all obviously committing a lot of our own 
time and energy to this, and we need to work well together for it to be 
successful in the long term.

Brandt
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/HJHZPEALHSFWBBBZNF4XLGDHR5CXY7RE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

2021-04-03 Thread Terry Reedy 

On 4/3/2021 7:15 PM, Miro Hrončok wrote:

On 03. 04. 21 21:44, Łukasz Langa wrote:
The memory layout of PyThreadState was unintentionally changed in the 
recent 3.9.3 bugfix release. This leads to crashes on 32-bit systems 
when importing binary extensions compiled for Python 3.9.0 - 3.9.2. 
This is a regression.


We will be releasing a hotfix 3.9.4 around 24 hours from now to 
address this issue and restore ABI compatibility with C extensions 
built for Python 3.9.0 - 3.9.2.


Thanks for the hotifx.

However, I need to ask: Would this also happen if there was a rc version 
of 3.9.3?


Unless the mistake was just introduced, the mistake would have happened. 
 One this severe would likely have been caught within the week or two 
before a final.  But as Łukasz noted when announcing the change, .rcs 
are generally ignored.  (I suspect that most everyone assumes that 
someone else will test them.  And begging people to not do that does not 
work well enough to justify the release.) 3.8.5 (2020 July 20 was hotfix 
for 3.8.4 (2020 July 14, which did have a candidate, which did not get 
tested the way that 3.8.4 itself was.


--
Terry Jan Reedy


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/IN4YTIRP2SE6NGT4GCZXFH6PVNSGU23T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

2021-04-03 Thread Gregory P. Smith 
On Sat, Apr 3, 2021 at 7:49 PM Terry Reedy  wrote:

> On 4/3/2021 7:15 PM, Miro Hrončok wrote:
> > On 03. 04. 21 21:44, Łukasz Langa wrote:
> >> The memory layout of PyThreadState was unintentionally changed in the
> >> recent 3.9.3 bugfix release. This leads to crashes on 32-bit systems
> >> when importing binary extensions compiled for Python 3.9.0 - 3.9.2.
> >> This is a regression.
> >>
> >> We will be releasing a hotfix 3.9.4 around 24 hours from now to
> >> address this issue and restore ABI compatibility with C extensions
> >> built for Python 3.9.0 - 3.9.2.
> >
> > Thanks for the hotifx.
> >
> > However, I need to ask: Would this also happen if there was a rc version
> > of 3.9.3?
>
> Unless the mistake was just introduced, the mistake would have happened.
>   One this severe would likely have been caught within the week or two
> before a final.  But as Łukasz noted when announcing the change, .rcs
> are generally ignored.  (I suspect that most everyone assumes that
> someone else will test them.  And begging people to not do that does not
> work well enough to justify the release.) 3.8.5 (2020 July 20 was hotfix
> for 3.8.4 (2020 July 14, which did have a candidate, which did not get
> tested the way that 3.8.4 itself was.
>
> --
> Terry Jan Reedy
>

For 3.9.4 I suggest a strict revert of the offending change. I created such
a PR and attached it to the bpo-43710 issue. It is a holiday weekend for a
large swath of the world. The recursion based crasher issue the original
change was fixing can be saved for a future release and not made under time
pressure.

I filed https://bugs.python.org/issue43725 to track one suggested way to
help automate prevention of these from landing in a release branch and
slipping through the cracks in a release. (discuss that on the issue, not
here)

-Greg
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/LOZUUVC22ZCTSQDJEVCYQACLHZ43FF73/
Code of Conduct: http://python.org/psf/codeofconduct/