[Python-Dev] Re: PEP 638: Syntactic macros

2020-09-28 Thread Nick Coghlan 
On Mon., 28 Sep. 2020, 1:22 am Marco Sulla, 
wrote:

> I like this, but IMHO adding a character at the end of the macro name
> (the exclamation mark), lowers readability.
> I'd prefer a character at the start of the macro, a character that is
> not used as an unary operator.
>

For usage, whether something is a macro or not should either be irrelevant
(when they're used as a more powerful function call or decorator), or else
entirely obvious from the way you use it (when they're defining a new
pseudo-statement), so it doesn't make sense to emphasize the syntactic
marker too much. Putting the marker gives the compiler the info and reader
the info they need without being too obtrusive (and also matches the way
Rust macros are used). Using "!" as a prefix operator is also effectively
already claimed by IPython for shell command execution, so I'd be surprised
if we ever used that spelling for anything else.

For the PEP itself, I'd like to see the bijection macro presented in the
abstract to give context for the specification section. While I've had my
doubts about macros in the past due to the inevitable evolution of
different "dialects" of Python, I think we already hit that point years ago
through sophisticated use of metaclasses and other features (Django code
looks very different from NumPy code, for example). Judicious use of macros
should offer opportunities to make domain specific dialects *easier* for
new users to pick up, rather than harder.

For the specification section, my main question/suggestion would be around
sibling macros: how about calling those "decorator macros" and making the
invocation syntax "@name!" rather than the bare "name!"?

Cheers,
Nick.






>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/6JLGAL5GMR7BXCLUK5J63UQOEABMXQ74/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 638: Syntactic macros

2020-09-28 Thread Marco Sulla 
On Mon, 28 Sep 2020 at 13:56, Nick Coghlan  wrote:
> For usage, whether something is a macro or not should either be irrelevant 
> (when they're used as a more powerful function call or
> decorator), or else entirely obvious from the way you use it (when they're 
> defining a new pseudo-statement), so it doesn't make sense to
> emphasize the syntactic marker too much.

IMHO `import!` can be easily confused with `import`, even with a
monospace font. An alternative is that macro must have a name
different from keywords (so you will have macro_import!, for example),
even if I'd prefer my first proposal (maybe something like %import).

> Putting the marker gives the compiler the info and reader the info they need 
> without being too obtrusive (and also matches the way Rust
> macros are used). Using "!" as a prefix operator is also effectively already 
> claimed by IPython for shell command execution, so I'd be
> surprised if we ever used that spelling for anything else.

Python uses None instead of Null and try-except instead of try-catch :-)

PS: I agree with the rest of the post.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/H5N5YE5JYSZZTR73CJLJNUTWRDPRYN2R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why do binary arithmetic operators care about differing method implementations but rich comparisons don't?

2020-09-28 Thread Eric Wieser 
Since I don't see it linked anywhere here: this was discussed a few years ago 
at https://bugs.python.org/issue30140.

Eric
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/A3CWMCKIOHU7S6O4SHZULOQWLZKVQG7C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why do binary arithmetic operators care about differing method implementations but rich comparisons don't?

2020-09-28 Thread Brett Cannon 
On Sun, Sep 27, 2020 at 9:56 PM Guido van Rossum  wrote:

> On Sun, Sep 27, 2020 at 5:58 PM Brett Cannon  wrote:
>
>>
>>
>> On Sun, Sep 27, 2020 at 2:58 PM Guido van Rossum 
>> wrote:
>>
>>> Hm... IIRC the reason why we did this for `__r*__` is because the more
>>> derived class might want to return an instance of that class, and we can't
>>> assume that the less derived class knows how to create an instance of the
>>> more derived class (the `__init__` signatures might differ).
>>>
>>
>> Yep, that's what the data model docs suggest (see the note at
>> https://docs.python.org/3/reference/datamodel.html#object.__ror__).
>>
>> But the interesting bit is skipping the call of __r*__ when `lhs.__r*__
>> == rhs.__r*__` (as long as the derived class requirements are met). That's
>> the difference that I'm really curious about compared to rich comparisons
>> and their inverse which don't have this call avoidance.
>>
>> To help visualize all of this, you can see
>> https://github.com/brettcannon/desugar/blob/066f16c00a2c78784bfb18eec31476df045cefe5/desugar/operator.py#L93-L103
>> for binary arithmetic operators compared to
>> https://github.com/brettcannon/desugar/blob/066f16c00a2c78784bfb18eec31476df045cefe5/desugar/operator.py#L273-L280
>> for rich comparisons.
>>
>
> Ooh, interesting. (Aren't you missing a few checks for MISSING in the elif
> or else branches?)
>

Nope, I handle that generically in the `for` loop farther down that makes
the actual calls. That one _MISSING check is because since it's just an
instance of `object()` then that subclass check will always succeed. I
should probably just define a custom singleton class to let me drop that
one guard case.


>
> Let me guess some more (I'm on a rare caffeine high since 9am so we'll see
> how this goes :-).
>
> The idea is clearly that if lhs and rhs are the same class we don't bother
> calling `__r*__` (because if `__*__` didn't do it there's no reason that
> `__r*__` would be any different).
>

That was my assumption.


>
> Are you sure you read things right, and `__r*__` is skipped when the
> `__r*__` methods are the same, and not only when the lhs and rhs classes
> are the same?
>

The test I wrote for this is at
https://github.com/brettcannon/desugar/blob/066f16c00a2c78784bfb18eec31476df045cefe5/tests/operator/test_binary_arithmetic.py#L92-L105
and passes when run against CPython via the 'operator' module which just
delegates to the syntax anyway.

And the code that makes this happen is (I think)
https://github.com/python/cpython/blob/6f8c8320e9eac9bc7a7f653b43506e75916ce8e8/Objects/abstract.c#L797-L798
.

BTW I have this all linked and written down in
https://snarky.ca/unravelling-binary-arithmetic-operations-in-python/.


>
> It does seem kind of a pointless optimization, since if the first call is
> successful we'll skip the second call anyway, and if it returns
> NotImplemented, well, if our assumption that `__r*__` is going to do the
> same, it's going to be an error anyway. I wonder if this was always there?
> Maybe we should study the git blame some more.
>

Assuming I have the write line of C doe, it looks like you introduced it in
2.2 with new-style classes, 19 years ago to the day. 😄
https://github.com/python/cpython/commit/4bb1e36eec19b30f2e582fceffa250e1598262e1


>
> And why don't we do this for rich comparisons? Probably because the logic
> is completely separate. :-( And maybe when we did rich comparisons (nearly
> a decade after the original binary operator overloading IIRC) the
> optimization idea didn't occur to us, or maybe we realized that we'd be
> optimizing an error case. Or maybe because rich comparisons were trying to
> somehow model the earlier `__cmp__`?
>

My guess was no one honestly knew/remembered this quirk existed for binary
arithmetic operators who were involved with rich comparisons.


>
>
>> [SNIP]
>>> I think we could try to change it but it would require a very careful
>>> risk analysis.
>>>
>>
>> I'm not sure how critical it is to change. I'm sure there's some
>> potential perf gain by avoiding the (potentially) unnecessary call, but I
>> also don't know if people have implemented these functions in such a way
>> that skipping the inverse operation on the right-hand side object would
>> break something. Would abuse of the syntax make a difference (e.g. making
>> `>` do something magical)?
>>
>
> I don't know, PEP 207 explicitly says the reflexivity assumptions are
> assumed. I guess I misunderstood your question for clarification as a
> suggestion to change.
>

I'm just looking for historical context for a blog post is all. If we feel
it's worth considering making the logic more uniform across operators then
I think that's worth considering, but I am personally okay considering this
a historical quirk that this difference exists to begin with.


>
> I feel this requires more careful thought than I can muster tonight.
>

😄 Yeah, this is definitely digging into the bowels of Python.

-Brett


>
>
>> -Brett
>>
>>
>

[Python-Dev] Re: PEP 638: Syntactic macros

2020-09-28 Thread Emily Bowman 
On Mon, Sep 28, 2020 at 7:23 AM Marco Sulla 
wrote:

> On Mon, 28 Sep 2020 at 13:56, Nick Coghlan  wrote:
> > For usage, whether something is a macro or not should either be
> irrelevant (when they're used as a more powerful function call or
> > decorator), or else entirely obvious from the way you use it (when
> they're defining a new pseudo-statement), so it doesn't make sense to
> > emphasize the syntactic marker too much.
>
> IMHO `import!` can be easily confused with `import`, even with a
> monospace font. An alternative is that macro must have a name
> different from keywords (so you will have macro_import!, for example),
> even if I'd prefer my first proposal (maybe something like %import).
>

I think the whole point is that import! won't be obvious unless you're
looking for it, and those macros won't be obvious unless you're looking for
them. They're meant to blend in rather than stand out.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/DBM47OMSR5M4NYXERG6OV3BN332H2KCU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why do binary arithmetic operators care about differing method implementations but rich comparisons don't?

2020-09-28 Thread Guido van Rossum 
On Mon, Sep 28, 2020 at 12:03 PM Brett Cannon  wrote:

> And the code that makes this happen is (I think)
> https://github.com/python/cpython/blob/6f8c8320e9eac9bc7a7f653b43506e75916ce8e8/Objects/abstract.c#L797-L798
> .
>

Ah, that's much clearer than all the English words written so far here. :-)
Let me go over this function (binary_op1()) for subtraction, the example
from your blog.

One piece of magic is that there are no separate `__sub__` and `__rsub__`
implementations at this level -- the `tp_as_number` struct just has a slot
named `nb_subtract` that takes two objects and either subtracts them or
returns NotImplemented.

This means that (**at this level**) there really is no point in calling
`__rsub__` if the lhs and rhs have the same type, because it would
literally just call the same `nb_subtract` function with the same arguments
a second time.

And if the types are different but the functions in `nb_subtract` are still
the same, again we'd be calling the same function with the same arguments
twice.

The `nb_subtract` slot for Python classes dispatches to either `__sub__` or
`__rsub__` in a complicated way. The code is SLOT1BINFULL in typeobject.c,
which echoes binary_op1():
https://github.com/python/cpython/blob/b0dfc7581697f20385813582de7e92ba6ba0105f/Objects/typeobject.c#L6340-L6383

That's some macro!

Now, interestingly, this macro may call *both* `left.__sub__(right)` and
`right.__rsub__(left)`. That is surprising, since there's also logic to
call left's nb_subtract and right's nb_subtract in binary_op1(). What's up
with that? Could we come up with an example where `a-b` makes more than two
calls? For that to happen we'd have to trick binary_op1() into calling
both. But I think that's impossible, because all Python classes have the
same identical function in nb_subtract (the function is synthesized using
SLOT1BIN -> SLOT1BINFULL), and in that case binary_op1() skips the second
call (the two lines that Brett highlighted!). So we're good here.

But maybe here we have another explanation for why binary_op1() is careful
to skip the second call. (The slot function duplicates this logic so it
will only call `__sub__` in this case.)

Since rich comparison doesn't have this safeguard, can we trick *that* into
making more than two calls? No, because the "reverse" logic
(`self.__lt__(other)` -> `other.__gt__(self)` etc.) is only implemented
once, in do_richcompare() in abstract.c. The slot function in typeobject.c
(slot_tp_richcompare()) is totally tame.

So the difference goes back to the design at the C level -- the number
slots don't have separate `__sub__` and `__rsub__` implementations (the C
function in nb_subtract has no direct way of knowing if it was called on
behalf of its first or second argument), and the complications derive from
that. The rich comparison slot has a clear `op` flag that always tells it
which operation was requested, and the implementation is much saner because
of it.

So yes, in a sense the difference is because rich comparison is much newer
than binary operators in Python -- binary operators are still constrained
by the original design, which predates operator overloading in user code
(i.e. `__sub__` and `rsub__`). But it was not a matter of forgetting
anything -- it was a matter of better design.

(Brett, maybe this warrants an update to your blog post?)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/K2FVPKP3MYW7T2N4U7VAWAPHFVTJMEM7/
Code of Conduct: http://python.org/psf/codeofconduct/