Re: [Python-Dev] libxml2 installation/binding issue

2018-02-06 Thread Steve Holden 
On Mon, Feb 5, 2018 at 10:55 PM, Ethan Smith  wrote:

> This list is for the discussion of development *of* Python. For
> discussion of development *with* Python, you want python-list.
>

 ... Whose web page can be found at
https://mail.python.org/mailman/listinfo/python-list

Steve Holden
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

2018-02-06 Thread Steve Holden 
On Tue, Feb 6, 2018 at 3:40 AM, Steven D'Aprano  wrote:

> On Sat, Feb 03, 2018 at 11:45:15AM +0100, asrp wrote:
>
> ​[...]
>


> Here's a faked session showing the sort of thing I am referring to.
> (Note that this is just an example, not a proposal for a new language
> feature.)
>
> for x in [1, 2, 3]:
> print(x)
> otherwise:
> print("nothing there")
>
>
> prints 1, 2, 3.
>

​I think you meant

for x in ([1, 2, 3], ):

​
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Lukasz Langa 
To add a counter-example that I'm painfully familiar with: the old Thrift for 
Python makes its (mutable) structures hashable. This is "useful" because you 
can memoize functions that take Thrift structures as arguments. You can key 
dictionaries with them. And so on.

Unfortunately, more often then not this same structure is passed around and 
inevitably gets mutated in place. Everything breaks. What should be memoized 
isn't (and in pathological cases, what shouldn't be memoized is), dictionary 
access *using the same object* raises unexpected key errors but iterating on 
the dictionary reveals the key.

It's clearly clowntown and in hindsight we shouldn't have enabled this 
behavior. But we can't go back since too much code relies on hashability now 
and it's "mostly" fine.

As a solution, the new asyncio-only Thrift implementation for Python uses 
C-level structures to make sure they are truly immutable in Python. We hash 
them and cache them like there's no tomorrow.

- Ł


> On Feb 5, 2018, at 11:54 PM, Ivan Levkivskyi  wrote:
> 
> Just wanted to add my 5 cents here. I am a bit surprised how people are 
> scared by adding `__hash__` to mutable classes.
> From my experience it is quite normal, I was always thinking about `hash()` 
> as hashing a _value_,
> and never as hashing _identity_, if I would need the latter, there is a 
> different function for this, `id()`.
> Moreover, I often did this in situations where dataclasses would be useful: a 
> class with several fields,
> necessary dunders, and few other methods (record-like classes).
> My motivation was mostly speed-up by memorization.
> To be fair my use cases were mostly about some tree-like strictures, but this 
> is probably a coincidence.
> 
> FWIW it is not a super-safe practice, but neither super-dangerous.
> 
> --
> Ivan
> 
> 
> 
> On 5 February 2018 at 22:56, Nick Coghlan  > wrote:
> On 6 February 2018 at 03:47, Guido van Rossum  > wrote:
> > If there's going to be an API for it, it should be in the class, not
> > something that mutates the class afterwards.
> 
> Something I realised after posting the __class__ setting idea is that
> you can actually use a comparable trick to inject an unsafe hash from
> the frozen version into the mutable version:
> 
> >>> from dataclasses import dataclass
> >>> @dataclass
> ... class Example:
> ... a: int
> ... b: int
> ...
> >>> c = Example(1, 2)
> >>> hash(c)
> Traceback (most recent call last):
>  File "", line 1, in 
> TypeError: unhashable type: 'Example'
> 
> >>> @dataclass(frozen=True)
> ... class LockedExample(Example):
> ... pass
> ...
> >>> Example.__hash__ = LockedExample.__hash__
> >>> hash(c)
> 3713081631934410656
> 
> So "unsafe_hash=True" would just be a shorthand spelling of that which
> skips creating the full frozen version of the class (and with the
> explicit parameter, we can better document the risks of making
> something hashable without also freezing it post-creation).
> 
> Cheers,
> Nick.
> 
> --
> Nick Coghlan   |   [email protected]    |   
> Brisbane, Australia
> ___
> Python-Dev mailing list
> [email protected] 
> https://mail.python.org/mailman/listinfo/python-dev 
> 
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/levkivskyi%40gmail.com 
> 
> 
> ___
> Python-Dev mailing list
> [email protected] 
> https://mail.python.org/mailman/listinfo/python-dev 
> 
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl 
> 


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Steven D'Aprano 
On Mon, Feb 05, 2018 at 10:50:21AM -0800, David Mertz wrote:

> Absolutely I agree. 'unsafe_hash' as a name is clear warning to users.

(I don't mean to pick on David specifically, I had to reply to some 
message in this thread and I just picked his.)

I'm rather gobsmacked at the attitudes of many people here about hashing 
data classes. I thought *I* was the cynical pessimist who didn't have a 
high opinion of the quality of the average programmer, but according to 
this thread apparently I'm positively Pollyanna-esque for believing that 
most people will realise that if an API offers separate switches for 
hashable and frozen, you need to set *both* if you want both.

Greg Smith even says that writing dunders apart from __init__ is a code 
smell, and warns people not to write dunders. Seriously? I get that 
__hash__ is hard to write correctly, which is why we have a hash=True to 
do the hard work for us, but I can't help feeling that at the point 
we're saying "don't write dunders, any dunder, you'll only do it wrong" 
we have crossed over to the wrong side of the pessimist/optimist line.

But here we are: talking about naming a perfectly reasonable argument 
"unsafe_hash". Why are we trying to frighten people?

There is nothing unsafe about a DataClass with hash=True, frozen=True, 
but this scheme means that even people who know what they're doing will 
write unsafe_hash=True, frozen=True, as if hashability was some sort of 
hand grenade waiting to go off.

Perhaps we ought to deprecate __hash__ and start calling it 
__danger_danger_hash__ too? No, I don't think so.

In the past, we've (rightly!) rejected proposals to call things like 
eval "unsafe_eval", and that really is dangerously unsafe when used 
naively with untrusted, unsanitised data. Hashing mutable objects by 
accident might be annoyingly difficult and frustrating to debug, but 
code injection attacks can lead to identity theft and worse, serious 
consequences for real people.

I'm 100% in favour of programmer education, but I think this label is 
*miseducation*. We're suggesting that hashability is unsafe, regardless 
of whether the object is frozen or not.

I'd far prefer to get a runtime warning:

"Are you sure you want hash=True without frozen=True?"

(or words to that extent) rather than burden all uses of the hash 
parameter, good and bad, with the unsafe label.


-- 
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Guido van Rossum 
Where do you get the impression that one would have to explicitly request
__hash__ if frozen=True is set? To the contrary, my proposal is for
@dataclass to automatically add a __hash__ method when frozen=True is set.
This is what the code currently released as 3.7.0b1 does if hash=None (the
default).

On Tue, Feb 6, 2018 at 9:26 AM, Steven D'Aprano  wrote:

> On Mon, Feb 05, 2018 at 10:50:21AM -0800, David Mertz wrote:
>
> > Absolutely I agree. 'unsafe_hash' as a name is clear warning to users.
>
> (I don't mean to pick on David specifically, I had to reply to some
> message in this thread and I just picked his.)
>
> I'm rather gobsmacked at the attitudes of many people here about hashing
> data classes. I thought *I* was the cynical pessimist who didn't have a
> high opinion of the quality of the average programmer, but according to
> this thread apparently I'm positively Pollyanna-esque for believing that
> most people will realise that if an API offers separate switches for
> hashable and frozen, you need to set *both* if you want both.
>
> Greg Smith even says that writing dunders apart from __init__ is a code
> smell, and warns people not to write dunders. Seriously? I get that
> __hash__ is hard to write correctly, which is why we have a hash=True to
> do the hard work for us, but I can't help feeling that at the point
> we're saying "don't write dunders, any dunder, you'll only do it wrong"
> we have crossed over to the wrong side of the pessimist/optimist line.
>
> But here we are: talking about naming a perfectly reasonable argument
> "unsafe_hash". Why are we trying to frighten people?
>
> There is nothing unsafe about a DataClass with hash=True, frozen=True,
> but this scheme means that even people who know what they're doing will
> write unsafe_hash=True, frozen=True, as if hashability was some sort of
> hand grenade waiting to go off.
>
> Perhaps we ought to deprecate __hash__ and start calling it
> __danger_danger_hash__ too? No, I don't think so.
>
> In the past, we've (rightly!) rejected proposals to call things like
> eval "unsafe_eval", and that really is dangerously unsafe when used
> naively with untrusted, unsanitised data. Hashing mutable objects by
> accident might be annoyingly difficult and frustrating to debug, but
> code injection attacks can lead to identity theft and worse, serious
> consequences for real people.
>
> I'm 100% in favour of programmer education, but I think this label is
> *miseducation*. We're suggesting that hashability is unsafe, regardless
> of whether the object is frozen or not.
>
> I'd far prefer to get a runtime warning:
>
> "Are you sure you want hash=True without frozen=True?"
>
> (or words to that extent) rather than burden all uses of the hash
> parameter, good and bad, with the unsafe label.
>
>
> --
> Steve
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Ethan Furman 

On 02/06/2018 09:38 AM, Guido van Rossum wrote:


Where do you get the impression that one would have to explicitly request 
__hash__ if frozen=True is set? To the
contrary, my proposal is for @dataclass to automatically add a __hash__ method 
when frozen=True is set. This is what the
code currently released as 3.7.0b1 does if hash=None (the default).


Which is my issue with the naming -- although, really, it's more with the 
parameter/argument:  in a hand-written class,

  __hash__ = None

means the object in is not hashable, but with the decorator:

  @dataclass(..., hash=None, ...)

it means something else.

My preference for "fixing" the issue:

1) make the default be a custom object (not None), so that `hash=None`
   means disable hashing

2) change the param name -- maybe to `add_hash` (I agree with D'Aprano
   that `unsafe_hash` can be misleading)

--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Guido van Rossum 
We may be in violent agreement.

I propose *not* to add a way to *disable* hashing when the rest of the
flags to @dataclass() would indicate that it's safe to add a __hash__
method.

I propose that with @dataclass(unsafe_hash=False) (the default), a __hash__
method is added when the following conditions are true:

- frozen=True (not the default)
- compare=True (the default)
- no __hash__ method is defined in the class

If we instead use @dataclass(unsafe_hash=True), a __hash__ will be added
regardless of the other flags, but if a __hash__ method is present, an
exception is raised.

Other values (e.g. unsafe_hash=None) are illegal for this flag.

Note that the the hash= flag to the field() function is unchanged from
what's currently in PEP 557 or in the implementation in 3.7.0b1. In
particular, the generated __hash__ method will disregard fields declared
using field(hash=False). It will also disregard fields declared using
field(compare=False, hash=False|None).


On Tue, Feb 6, 2018 at 10:11 AM, Ethan Furman  wrote:

> On 02/06/2018 09:38 AM, Guido van Rossum wrote:
>
> Where do you get the impression that one would have to explicitly request
>> __hash__ if frozen=True is set? To the
>> contrary, my proposal is for @dataclass to automatically add a __hash__
>> method when frozen=True is set. This is what the
>> code currently released as 3.7.0b1 does if hash=None (the default).
>>
>
> Which is my issue with the naming -- although, really, it's more with the
> parameter/argument:  in a hand-written class,
>
>   __hash__ = None
>
> means the object in is not hashable, but with the decorator:
>
>   @dataclass(..., hash=None, ...)
>
> it means something else.
>
> My preference for "fixing" the issue:
>
> 1) make the default be a custom object (not None), so that `hash=None`
>means disable hashing
>
> 2) change the param name -- maybe to `add_hash` (I agree with D'Aprano
>that `unsafe_hash` can be misleading)
>
> --
> ~Ethan~
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Ethan Furman 

On 02/06/2018 11:18 AM, Guido van Rossum wrote:


We may be in violent agreement.

I propose *not* to add a way to *disable* hashing when the rest of the flags to 
@dataclass() would indicate that it's
safe to add a __hash__ method.


Okay.


I propose that with @dataclass(unsafe_hash=False) (the default), a __hash__ 
method is added when the following
conditions are true:

- frozen=True (not the default)
- compare=True (the default)
- no __hash__ method is defined in the class

If we instead use @dataclass(unsafe_hash=True), a __hash__ will be added 
regardless of the other flags, but if a
__hash__ method is present, an exception is raised.

Other values (e.g. unsafe_hash=None) are illegal for this flag.


Ah!  Excellent, that greatly allays my worries.


Note that the the hash= flag to the field() function is unchanged from what's 
currently in PEP 557 or in the
implementation in 3.7.0b1. In particular, the generated __hash__ method will 
disregard fields declared using
field(hash=False). It will also disregard fields declared using 
field(compare=False, hash=False|None).


It sounds like `unsafe_hash=True` indicates a truly unsafe hash (that is, mutable data is involved in the hash 
calculation), but there still seems to be one possibility for an "unsafe_hash" to actually be safe -- that is, if only 
immutable fields are used in __eq__, then dataclass could safely generate a hash for us.


Do we have a way to know if the equality fields are hashable?  I suppose we could check each one for a for a non-None 
__hash__.  Then we could modify that first condition from


- frozen=True

to

- frozen=True or all(getattr(eq_fld, '__hash__', None) is not None for eq_field 
in equality_fields)

Thoughts?


On a different note, should the PEP be updated with the current signature?  It still talks about hash=None being the 
default.


--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread David Mertz 
Honestly, the name I would most want for the keyword argument is '_hash'.
That carries the semantics I desire.

On Feb 6, 2018 10:13 AM, "Ethan Furman"  wrote:

> On 02/06/2018 09:38 AM, Guido van Rossum wrote:
>
> Where do you get the impression that one would have to explicitly request
>> __hash__ if frozen=True is set? To the
>> contrary, my proposal is for @dataclass to automatically add a __hash__
>> method when frozen=True is set. This is what the
>> code currently released as 3.7.0b1 does if hash=None (the default).
>>
>
> Which is my issue with the naming -- although, really, it's more with the
> parameter/argument:  in a hand-written class,
>
>   __hash__ = None
>
> means the object in is not hashable, but with the decorator:
>
>   @dataclass(..., hash=None, ...)
>
> it means something else.
>
> My preference for "fixing" the issue:
>
> 1) make the default be a custom object (not None), so that `hash=None`
>means disable hashing
>
> 2) change the param name -- maybe to `add_hash` (I agree with D'Aprano
>that `unsafe_hash` can be misleading)
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mertz%
> 40gnosis.cx
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Eric V. Smith 

On 2/6/18 2:40 PM, Ethan Furman wrote:

On a different note, should the PEP be updated with the current
signature?  It still talks about hash=None being the default.


Once we've reached an agreement, I'll update the PEP. I don't think 
we're there quite yet.


Eric
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Guido van Rossum 
That's much less self-descriptive and harder to search Google or
StackOverflow for. It's also easier to overlook. We really want to send the
signal that this is unsafe and requires serious consideration before it is
turned on.

On Tue, Feb 6, 2018 at 11:57 AM, David Mertz  wrote:

> Honestly, the name I would most want for the keyword argument is '_hash'.
> That carries the semantics I desire.
>
> On Feb 6, 2018 10:13 AM, "Ethan Furman"  wrote:
>
>> On 02/06/2018 09:38 AM, Guido van Rossum wrote:
>>
>> Where do you get the impression that one would have to explicitly request
>>> __hash__ if frozen=True is set? To the
>>> contrary, my proposal is for @dataclass to automatically add a __hash__
>>> method when frozen=True is set. This is what the
>>> code currently released as 3.7.0b1 does if hash=None (the default).
>>>
>>
>> Which is my issue with the naming -- although, really, it's more with the
>> parameter/argument:  in a hand-written class,
>>
>>   __hash__ = None
>>
>> means the object in is not hashable, but with the decorator:
>>
>>   @dataclass(..., hash=None, ...)
>>
>> it means something else.
>>
>> My preference for "fixing" the issue:
>>
>> 1) make the default be a custom object (not None), so that `hash=None`
>>means disable hashing
>>
>> 2) change the param name -- maybe to `add_hash` (I agree with D'Aprano
>>that `unsafe_hash` can be misleading)
>>
>> --
>> ~Ethan~
>> ___
>> Python-Dev mailing list
>> [email protected]
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mertz%40g
>> nosis.cx
>>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Guido van Rossum 
On Tue, Feb 6, 2018 at 11:40 AM, Ethan Furman  wrote:

> It sounds like `unsafe_hash=True` indicates a truly unsafe hash (that is,
> mutable data is involved in the hash calculation), but there still seems to
> be one possibility for an "unsafe_hash" to actually be safe -- that is, if
> only immutable fields are used in __eq__, then dataclass could safely
> generate a hash for us.
>
> Do we have a way to know if the equality fields are hashable?  I suppose
> we could check each one for a for a non-None __hash__.  Then we could
> modify that first condition from
>
> - frozen=True
>
> to
>
> - frozen=True or all(getattr(eq_fld, '__hash__', None) is not None for
> eq_field in equality_fields)
>

There seems to be a misunderstanding underlying these questions. Even if
all fields have an immutable type (e.g. all ints, supporting __eq__ and
__hash__), if the containing class isn't frozen, they can be assigned to.
E.g.

@dataclass()
class Point:
x: int
y: int

p = Point(1, 1)
p.x = 2  # This is legal

The only way to make that assignment to p.x illegal is to make the *class*
frozen (using @dataclass(frozen=True)) -- nothing we can do about the
*field* will change this.

Of course if you use @dataclass(frozen=True, unsafe_hash=True) you may
still get a safe hash. :-)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Ethan Furman 

On 02/06/2018 12:24 PM, Guido van Rossum wrote:

On Tue, Feb 6, 2018 at 11:40 AM, Ethan Furman wrote:



It sounds like `unsafe_hash=True` indicates a truly unsafe hash (that is,

>> mutable data is involved in the hash calculation), but there still seems
>> to be one possibility for an "unsafe_hash" to actually be safe -- that is,
>> if only immutable fields are used in __eq__, then dataclass could safely
>> generate a hash for us.


Do we have a way to know if the equality fields are hashable?  I suppose

>> we could check each one for a for a non-None __hash__.  Then we could
>> modify that first condition from


- frozen=True

to

- frozen=True or all(getattr(eq_fld, '__hash__', None) is not None for

>>   eq_field in equality_fields)


There seems to be a misunderstanding underlying these questions. Even if

> all fields have an immutable type (e.g. all ints, supporting __eq__ and
> __hash__), if the containing class isn't frozen, they can be assigned to.
> E.g.


@dataclass()
class Point:
  x: int
  y: int

p = Point(1, 1)
p.x = 2  # This is legal

The only way to make that assignment to p.x illegal is to make the *class*

> frozen (using @dataclass(frozen=True)) -- nothing we can do about the *field*
> will change this.

Oh, right.  When I was thinking this I thought a field could be frozen individually, didn't find the option at the field 
level when I checked the PEP, and then promptly forgot and suggested it anyway.


Although, couldn't we add a field-level frozen attribute (using property for the implementation), and check that all 
equality fields are properties as well as hashable?


--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Guido van Rossum 
On Tue, Feb 6, 2018 at 12:44 PM, Ethan Furman  wrote:

> Although, couldn't we add a field-level frozen attribute (using property
> for the implementation), and check that all equality fields are properties
> as well as hashable?
>

That would be a totally unrelated feature request. Let's wait whether it's
actually needed a lot before designing it.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

2018-02-06 Thread asrp 
> Message-ID: <[email protected]>
> 
> On Sat, Feb 03, 2018 at 11:45:15AM +0100, asrp wrote:
> 
> > > Can you give an example of how you would do that? I don't mean the 
> > > mechanism used, I mean how would a developer implement a new syntactic 
> > > feature. Suppose I wanted to add a new clause to for...else, let's say:
> > >
> > > for ... :
> > >block
> > > otherwise:
> > ># runs only if the for-loop was empty
> > > 
> > > How would do I do that?
> [...]
> > If you tell me a bit more about the intended behaviour of "otherwise", 
> > I'd be happy to do an example with that clause.
> 
> 
> Here's a faked session showing the sort of thing I am referring to. 
> (Note that this is just an example, not a proposal for a new language 
> feature.)
> 
> for x in [1, 2, 3]:
> print(x)
> otherwise:
> print("nothing there")
> 
> 
> prints 1, 2, 3.
> 
> for x in []:
> print(x)
> otherwise:
> print("nothing there")
> 
> prints "nothing there". In other words, the otherwise block runs if, and 
> only if, the loop iterable is empty and the for block does NOT run.
> 
> Can you do something like that?
> 

Oh, I see. Yes, definitely. This time I'll change for_stmt in lib/simple_ast.py 
beforehand (but runtime reload also works).

def for_stmt(index_var, iterable, block, else_block, otherwise_block):
iterator = iter(evaluate(iterable))
try:
assignment(index_var, iterator.next())
except StopIteration:
evaluate(otherwise_block)
return
while_true:
__caller__['__continue__'] = __continue__
__caller__['__break__'] = __break__
evaluate(block)
try:
assignment(index_var, iterator.next())
except StopIteration:
evaluate(else_block)
return

Then start the interpreter

$ ipython -i test/python_repl.py 
p>> simport simple_ast
p>> ^D
[...]
In [1]: grammar = python_grammar.full_definition + python_grammar.extra

In [2]: grammar += r"""
   ...: for_stmt = "for" {exprlist} "in" {testlist} ":" {suite} {((SAME_INDENT 
"else" ":" {suite}) | void=pass_stmt) ((SAME_INDENT "otherwise" ":" {suite}) | 
void=pass_stmt)}
   ...: """

In [3]: pyterp.parser = python.Interpreter(i3.parse("grammar", grammar))

In [4]: pyterp.repl()
p>> for x in [1, 2]:
...   print(x)
... otherwise:
...   print("Nothing there")
... 
1
2
p>> for x in []:
...   print(x)
... otherwise:
...   print("Nothing there")
... 
Nothing there
p>> for x in [1, 2]:
...   print(x)
... else:
...   print("Something there")
... otherwise:
...   print("Nothing there")
... 
1
2
Something there


I noticed since my last post that the else and otherwise blocks don't need to 
be named if they are positional in for_stmt. The grammar change here reflects 
that.

I've also posted an example with `until_stmt` and I talk a bit more about 
debugging changes made there.

http://blog.asrpo.com/adding_new_statement

(Nothing here is a proposed language change, just demos.)

asrp

> 
> 
> -- 
> Steve
> 
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

2018-02-06 Thread asrp 
> 
> 
> https://refi64.com/posts/the-magic-of-rpython.html
> 
> Note that this was written back when I used "like" and "really" in every
> sentence, and when I used to think that Python copied tuples (don't ask).
> 

Thanks! And thanks for making the RPython language a bit more explicit for the 
rest of us.

Reading your post reminded me that having nested lists of lists of ... 
eventually strings (not all at the same nesting level) already made it 
difficult. I don't remember if I stopped trying there or a bit more further 
along.

asrp

> On Feb 3, 2018 at 10:05 AM, > wrote:
> 
> > Are you aware of pypy?
> >
> 
> Yes, but I do not know it well. It seems to be more focused on JIT
> compiling with many of its functions written for special cases and I
> did not find a way to make experimental changes easily.
> 
> I even tried (briefly) to have the main loop of this interpreter run
> in RPython but didn't find enough information on which subset of
> Python does RPython handle.
> 
> 
> > Can you give an example of how you would do that? I don't mean the
> > mechanism used, I mean how would a developer implement a new syntactic
> > feature. Suppose I wanted to add a new clause to for...else, let's say:
> >
> > for ... :
> >block
> > otherwise:
> ># runs only if the for-loop was empty
> >
> > How would do I do that?
> >
> 
> Here's one way to do that. Sorry for the lack of doc for the moment.
> 
> To alter the language's grammar, you'd create a string with the new
> grammar, parse it and set the result to pyterp.parser. Then, depending
> on how the grammar was changed (which affects what AST is created),
> you'd change the corresponding function to handle the new semantics.
> 
> I didn't quite understand what you mean by "was empty". However, I
> noticed that "else" blocks after for are ignored in the current
> implement. Here's an example session adding "else".
> 
> If you tell me a bit more about the intended behaviour of "otherwise",
> I'd be happy to do an example with that clause.
> 
> $ ipython -i test/python_repl.py
> p>> simport simple_ast
> p>> for i in [1, 2]:
> ... print i
> ...
> 1
> 2
> p>> for i in [1, 2, 3]:
> ... print i
> ... else:
> ... print 100
> ...
> 1
> 2
> 3
> p>> ^D
> 
> In the above for loop, the else block is ignored! I press Control-D to
> exist the interpreter.
> 
> In [1]: grammar = python_grammar.full_definition + python_grammar.extra
> 
> In [2]: grammar += r"""
>...: for_stmt = "for" {exprlist} "in" {testlist} ":" {suite}
> {(SAME_INDENT "else" ":" {suite}) | void=pass_stmt}
>...: """
> 
> In [3]: pyterp.parser = python.Interpreter(i3.parse("grammar", grammar))
> 
> In [4]: pyterp.repl()
> 
> Now edit for_stmt in simple_ast.py
> 
>  except StopIteration:
> +evaluate(else_block)
>  return
> 
> (If else_block was not already an (ignored) parameter, we'd have to
> change the `def for_stmt...` line too.) And run in the REPL we just
> restarted.
> 
> p>> simple_ast.reload_module(simple_ast)
> p>> for i in [1, 2, 3]:
> ... print i
> ... else:
> ... print 100
> ...
> 1
> 2
> 3
> 100
> p>> for i in [1, 2, 3]:
> ... print i
> ... break
> ... else:
> ... print 100
> ...
> 1
> p>>
> 
> 
> 
> Some notes:
> 
> 1. I'm using the host Python interpreter to change the grammar here
> but that's not strictly necessary if we expose pyterp itself in the
> global scope.
> 2. I'm editing simple_ast.py and reloading because its less changes
> but redefining a new function and setting simple_ast.for_stmt (in
> other words, monkey-patching the change) should also work.
> 
> If we wanted an "otherwise" clause on top of an "else" clause, we
> probably want to make some provision for the runtime to distinguish
> between the two so I'd edit the grammar with something like
> 
> for_stmt = "for" {exprlist} "in" {testlist} ":" {suite}
> {((SAME_INDENT "else" ":" {suite}) | void=pass_stmt)=else_block
> ((SAME_INDENT "otherwise" ":" {suite}) |
> void=pass_stmt)=otherwise_block}
> 
> (In this case, "otherwise" blocks have to come after the "else" block
> if both are present.)
> 
> asrp
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Eric V. Smith 
Sorry for the late reply. Still recovering from a computer failure. 

My only concern with this approach is: what if you don’t want any __hash__ 
added? Say you want to use your base class’s hashing. I guess you could always 
“del cls.__hash__” after the class is created, but it’s not elegant. 

That’s what we got from the tri-state option: never add (False), always add 
(True), or add if it’s safe (None).

--
Eric

> On Feb 5, 2018, at 12:49 AM, Guido van Rossum  wrote:
> 
> Looks like this is turning into a major flamewar regardless of what I say. :-(
> 
> I really don't want to lose the ability to add a hash function to a mutable 
> dataclass by flipping a flag in the decorator. I'll explain below. But I am 
> fine if this flag has a name that clearly signals it's an unsafe thing to do.
> 
> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the 
> @dataclass decorator with a simpler flag named unsafe_hash=. This would be a 
> simple bool (not a tri-state flag like the current hash=None|False|True). The 
> default would be False, and the behavior then would be to add a hash function 
> automatically only if it's safe (using the same rules as for hash=None 
> currently). With unsafe_hash=True, a hash function would always be generated 
> that takes all fields into account except those declared using 
> field(hash=False). If there's already a `def __hash__` in the function I 
> don't care what it does, maybe it should raise rather than quietly doing 
> nothing or quietly overwriting it.
> 
> Here's my use case.
> 
> A frozen class requires a lot of discipline, since you have to compute the 
> values of all fields before calling the constructor. A mutable class allows 
> other initialization patterns, e.g. manually setting some fields after the 
> instance has been constructed, or having a separate non-dunder init() method. 
> There may be good reasons for using these patterns, e.g. the object may be 
> part of a cycle (e.g. parent/child links in a tree). Or you may just use one 
> of these patterns because you're a pretty casual coder. Or you're modeling 
> something external.
> 
> My point is that once you have one of those patterns in place, changing your 
> code to avoid them may be difficult. And yet your code may treat the objects 
> as essentially immutable after the initialization phase (e.g. a parse tree). 
> So if you create a dataclass and start coding like that for a while, and much 
> later you need to put one of these into a set or use it as a dict key, 
> switching to frozen=True may not be a quick option. And writing a __hash__ 
> method by hand may feel like a lot of busywork. So this is where 
> [unsafe_]hash=True would come in handy.
> 
> I think naming the flag unsafe_hash should take away most objections, since 
> it will be clear that this is not a safe thing to do. People who don't 
> understand the danger are likely to copy a worse solution from StackOverflow 
> anyway. The docs can point to frozen=True and explain the danger.
> 
> -- 
> --Guido van Rossum (python.org/~guido)
> 
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Guido van Rossum 
That seems a rare case (though I hadn't thought of it). I had thought of
the use case where you want a frozen type without a hash; that you can
presumably implement using

def __hash__(self): raise TypeError("not hashable")

We can do a similar thing to preserve the superclass __hash__ if it's rare
enough:

def __hash__(self): return super().__hash__()

If at all possible I'd like to kill the tri-state hash= flag -- the amount
of time spent creating and discussing the huge table in the bpo issue are
an indication of how much effort it would take other people to understand
it.

If either of those use cases becomes annoyingly common we'll have to think
of something else.

On Tue, Feb 6, 2018 at 5:38 PM, Eric V. Smith  wrote:

> Sorry for the late reply. Still recovering from a computer failure.
>
> My only concern with this approach is: what if you don’t want any __hash__
> added? Say you want to use your base class’s hashing. I guess you could
> always “del cls.__hash__” after the class is created, but it’s not elegant.
>
> That’s what we got from the tri-state option: never add (False), always
> add (True), or add if it’s safe (None).
>
> --
> Eric
>
> On Feb 5, 2018, at 12:49 AM, Guido van Rossum  wrote:
>
> Looks like this is turning into a major flamewar regardless of what I say.
> :-(
>
> I really don't want to lose the ability to add a hash function to a
> mutable dataclass by flipping a flag in the decorator. I'll explain below.
> But I am fine if this flag has a name that clearly signals it's an unsafe
> thing to do.
>
> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
> a simple bool (not a tri-state flag like the current hash=None|False|True).
> The default would be False, and the behavior then would be to add a hash
> function automatically only if it's safe (using the same rules as for
> hash=None currently). With unsafe_hash=True, a hash function would always
> be generated that takes all fields into account except those declared using
> field(hash=False). If there's already a `def __hash__` in the function I
> don't care what it does, maybe it should raise rather than quietly doing
> nothing or quietly overwriting it.
>
> Here's my use case.
>
> A frozen class requires a lot of discipline, since you have to compute the
> values of all fields before calling the constructor. A mutable class allows
> other initialization patterns, e.g. manually setting some fields after the
> instance has been constructed, or having a separate non-dunder init()
> method. There may be good reasons for using these patterns, e.g. the object
> may be part of a cycle (e.g. parent/child links in a tree). Or you may just
> use one of these patterns because you're a pretty casual coder. Or you're
> modeling something external.
>
> My point is that once you have one of those patterns in place, changing
> your code to avoid them may be difficult. And yet your code may treat the
> objects as essentially immutable after the initialization phase (e.g. a
> parse tree). So if you create a dataclass and start coding like that for a
> while, and much later you need to put one of these into a set or use it as
> a dict key, switching to frozen=True may not be a quick option. And writing
> a __hash__ method by hand may feel like a lot of busywork. So this is where
> [unsafe_]hash=True would come in handy.
>
> I think naming the flag unsafe_hash should take away most objections,
> since it will be clear that this is not a safe thing to do. People who
> don't understand the danger are likely to copy a worse solution from
> StackOverflow anyway. The docs can point to frozen=True and explain the
> danger.
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> eric%2Ba-python-dev%40trueblade.com
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Ethan Furman 

On 02/06/2018 06:48 PM, Guido van Rossum wrote:


That seems a rare case (though I hadn't thought of it). I had thought of the 
use case where you want a frozen type
without a hash; that you can presumably implement using

def __hash__(self): raise TypeError("not hashable")

We can do a similar thing to preserve the superclass __hash__ if it's rare 
enough:

def __hash__(self): return super().__hash__()

If at all possible I'd like to kill the tri-state hash= flag -- the amount of 
time spent creating and discussing the
huge table in the bpo issue are an indication of how much effort it would take 
other people to understand it.


I think the biggest reason this has become so complicated is because we are 
refusing to use an Enum:

class Hashable(Enum):
IF_SAFE = 1
ADD = 2
DEFER = 3
NONE = 4

IF_SAFE is currently the False value.
ADD is currently the True value
DEFER means don't add one
NONE means set __hash__ to None

The only thing missing now is a setting to indicate that dataclass should do nothing if the class already has a __hash__ 
method -- possibly DEFER, although I think IF_SAFE can include "the class already has one, it's not safe to override it".



If either of those use cases becomes annoyingly common we'll have to think of 
something else.


Or we could solve it now and not have to deal with backwards-compatibility 
issues in the future.

--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

2018-02-06 Thread Guido van Rossum 
I'm not a fan, sorry.

On Tue, Feb 6, 2018 at 7:33 PM, Ethan Furman  wrote:

> On 02/06/2018 06:48 PM, Guido van Rossum wrote:
>
> That seems a rare case (though I hadn't thought of it). I had thought of
>> the use case where you want a frozen type
>> without a hash; that you can presumably implement using
>>
>> def __hash__(self): raise TypeError("not hashable")
>>
>> We can do a similar thing to preserve the superclass __hash__ if it's
>> rare enough:
>>
>> def __hash__(self): return super().__hash__()
>>
>> If at all possible I'd like to kill the tri-state hash= flag -- the
>> amount of time spent creating and discussing the
>> huge table in the bpo issue are an indication of how much effort it would
>> take other people to understand it.
>>
>
> I think the biggest reason this has become so complicated is because we
> are refusing to use an Enum:
>
> class Hashable(Enum):
> IF_SAFE = 1
> ADD = 2
> DEFER = 3
> NONE = 4
>
> IF_SAFE is currently the False value.
> ADD is currently the True value
> DEFER means don't add one
> NONE means set __hash__ to None
>
> The only thing missing now is a setting to indicate that dataclass should
> do nothing if the class already has a __hash__ method -- possibly DEFER,
> although I think IF_SAFE can include "the class already has one, it's not
> safe to override it".
>
> If either of those use cases becomes annoyingly common we'll have to think
>> of something else.
>>
>
> Or we could solve it now and not have to deal with backwards-compatibility
> issues in the future.
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com