Re: [Python-ideas] PEP 532: A circuit breaking operator and protocol
(Hi, I'm a first-time poster. I was inspired by Raymond Hettinger's keynote
at PyCon CA to look at new PEPs and comment on them. Hopefully, I'm not
committing any faux-pas in my post!)
1) I don't think this proposal sufficiently handles falsy values.
What is the expected value of x in the following?
x = exists(0) else 1
Since the PEP specifically states that exists() is to check that the input
is not None, as a user, I would expect x == 0. However, from my
interpretation, it appears that the value 0 would still be directly
evaluated for truthiness and we would get x == 1.
I don't fully get what the purpose of __then__ and __else__ are meant to
be, but it seems like instead of this:
type(_lhs).__then__(_lhs) if _lhs else type(_lhs).__else__(_lhs, RHS)
you would want:
LHS if type(_lhs).__then__(_lhs) else RHS
Where __then__ returns a simple True/False result. (Maybe the name __then__
doesn't make sense in that case.)
2) My initial reaction was that `else` doesn't belong in an expression, but
I guess there's already precedent for that. (I actually wasn't aware of the
`x if y else z` expression until I read this PEP!)
I'm already not a fan of the overloading of else in the cases of for/else
and try/else. (Are there other uses? It's a hard thing to search on
Google...) Now, we're going to have `else`s that aren't anchored to other
statements. You've always known that an `else` belonged to the preceding
if/for/try at the same level of indentation. (Or in the same expression.)
Is there a chance of a missing colon silently changing the meaning of code?
if foo():
a = 1
else
bar()
(Probably not...)
Are two missing spaces too outrageous?
x = yiffoo() else bar()
I'm not 100% sure, but I think that a current parser would see that as a
syntax error very early, as opposed to having to wait for it to try to find
'yiffoo' at run-time.
3) Towards the end, you propose some very Perl-like syntax:
print(some_expensive_query()) if verbosity > 2
This seems completely unrelated to the rest of the PEP, and will likely
invite people to propose an `unless` operator if implemented. (Followed by
an `until` statement.) :)
While it does read more naturally like English, it seems superfluous in a
programming language.
4) The proposal shows how it fixes some common pain points:
value = missing(obj) else obj.field.of.interest
value = missing(obj) else obj["field"]["of"]["interest"]
But it doesn't address very similar ones:
missing(obj) else missing(obj.field) else missing(obj.field.of) else
obj.field.of.interest
obj.get('field', {}).get('of', {}).get('interest')
(The first example shows how it would be handled with the PEP in its
current state.)
Maybe these are too far out of scope, I'm not sure. They feel very similar
to me though.
I hope these are useful comments and not too nit-picky.
Thanks,
Ryan Fox
___
Python-ideas mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On Mon, Nov 14, 2016 at 01:19:30AM +0100, Mikhail V wrote: [...] > >> A good syntax example: > >> > >> a = sum (a, c) There is a reason why mathematicians and accountants use symbols as a compact notation for common functions, instead of purely functional notation. Here is a real-world example, the formula for compound interest: A = P*(1+r/100)**n Compare that to: A = mul(P, pow(sum(1, div(r, 100)), n)) Even Reverse Polish Notation is easier to read than that series of functions: P 1 r 100 / + n ** * Don't misunderstand me, functions are important, and over-use of cryptic symbols that are only meaningful to an expert will hurt readability and maintainability of code. But people have needed to do arithmetic for over six thousand years, and there is no good substitute for compact operators for basic operations. [...] > It is kind of clear from the context, that I am speaking of syntax and > not how things are working under the hood, or? > If a compiler cannot optimize "a = a + 1" into an in-place operation, > that is misfortune. That's not how Python works. Or at least, not without an extremely powerful and smart compiler, like PyPy. In Python, integers (and floats) are immutable objects. They have to be immutable, otherwise you would have things like this: x = 1 y = x x = x + 1 # changes the object 1 in place print(y*10) # expect 10, but get 20 That's how lists work, because they are mutable: py> x = [1] py> y = x py> x[0] = x[0] + 1 py> print(y[0]*10) # expect 1*10 = 10 20 A "sufficiently smart" compiler can work around this, as PyPy does under some circumstances, but you shouldn't expect this optimization to be simple. [...] > A better option would be to support unary operators so the user > can write directly without assignment: > > inc (a, 1) > > Would mean in-place increment "a" with 1 That is impossible with Python's executation model, in particular the "pass by object sharing" calling convention. I expect that this would require some extremely big changes to the way the compiler works, possibly a complete re-design, in order to allow pass by reference semantics. The biggest problem is that even if the compiler could choose between calling conventions, it would have to make that decision at runtime. It won't know until runtime that inc() requires pass by reference. That would make function calls even slower than they are now. -- Steve ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] str(slice(10)) should return "slice(10)"
On 13 November 2016 at 21:25, Ivan Levkivskyi wrote:
> This reminds me @ vs .dot() for matrix multiplication a bit. Pandas has
> IndexSlicer, NumPy has index_exp, etc. I think it would be nice to have a
> simple common way to express this.
> But here we have an additional ingredient -- generic types. I think that a
> reasonable compromise would be to simply continue the way proposed in
> http://bugs.python.org/issue24379 -- just add operator.subscript for this
> purpose. Pros:
> * subscript is not a class, so that subscript[...] will be not confused with
> generics;
> * this does not require patching built-ins;
> * all libraries that need this will get a "common interface" in stdlib,
> operator module seems to be good place for this.
>From an educational point of view, it also makes it a bit easier to
give interactive examples of how slicing syntax maps to the subscript
parameter on __getitem__, __setitem__ and __delitem__: you can just do
"print(operator.subscript[EXPR])" rather than having to build a dummy
__getitem__ implementation of your own.
If an actual use case is found for it, that approach would also leave
"operator.subscript('EXPR')" available for a micro-eval implementation
that evaluated a given string as a subscript rather than as a normal
top level expression.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-ideas mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 532: A circuit breaking operator and protocol
On 14 November 2016 at 19:01, Ryan Fox wrote: > (Hi, I'm a first-time poster. I was inspired by Raymond Hettinger's keynote > at PyCon CA to look at new PEPs and comment on them. Hopefully, I'm not > committing any faux-pas in my post!) > > 1) I don't think this proposal sufficiently handles falsy values. > > What is the expected value of x in the following? > > x = exists(0) else 1 > > Since the PEP specifically states that exists() is to check that the input > is not None, as a user, I would expect x == 0. However, from my > interpretation, it appears that the value 0 would still be directly > evaluated for truthiness and we would get x == 1. No, the conditional branching would be based on exists.__bool__ (or, in the current working draft, is_not_none.__bool__), and that would be "0 is not None", which would be True and hence short-circuit. > I don't fully get what the purpose of __then__ and __else__ are meant to be, > but it seems like instead of this: > > type(_lhs).__then__(_lhs) if _lhs else type(_lhs).__else__(_lhs, RHS) > > you would want: > > LHS if type(_lhs).__then__(_lhs) else RHS `__then__` is responsible for *unwrapping* the original value from the circuit breaker when it short-circuits: it's what allows the overall expression to return "0", even though the truth check is done based on "0 is not None". > Where __then__ returns a simple True/False result. (Maybe the name __then__ > doesn't make sense in that case.) We already have a method for that: __bool__. However, it has exactly the problem you describe, which is why "0 or expr" will always short-circuit, and why "(0 is not None) or expr" will return "True". > 2) My initial reaction was that `else` doesn't belong in an expression, but > I guess there's already precedent for that. (I actually wasn't aware of the > `x if y else z` expression until I read this PEP!) > > I'm already not a fan of the overloading of else in the cases of for/else > and try/else. (Are there other uses? It's a hard thing to search on > Google...) Now, we're going to have `else`s that aren't anchored to other > statements. You've always known that an `else` belonged to the preceding > if/for/try at the same level of indentation. (Or in the same expression.) > Is there a chance of a missing colon silently changing the meaning of code? > > if foo(): > a = 1 > else > bar() > > (Probably not...) No, due to Python's line continuation rules - you'd also need parentheses or a backslash to avoid getting a SyntaxError on the unfinished line. It does create amibiguities around conditional expressions though, hence why that comes up as one of the main pragmatic concerns with the idea. > Are two missing spaces too outrageous? > > x = yiffoo() else bar() > > I'm not 100% sure, but I think that a current parser would see that as a > syntax error very early, as opposed to having to wait for it to try to find > 'yiffoo' at run-time. That style of error is already possible with the other keyword based operators: x = notfoo() y = xorfoo() z = yandfoo() As you not, the main defense is that this will usually be a name error, picked up either at runtime or by a static code analyser. > 3) Towards the end, you propose some very Perl-like syntax: > > print(some_expensive_query()) if verbosity > 2 > > This seems completely unrelated to the rest of the PEP, and will likely > invite people to propose an `unless` operator if implemented. (Followed by > an `until` statement.) :) > > While it does read more naturally like English, it seems superfluous in a > programming language. De Morgan's laws [1] mean that 'and' and 'or' are technically redundant with each other, as given 'not', you can always express one in terms of the other: X and Y --> not ((not X) or (not Y)) X or Y --> not ((not X) and (not Y)) However, writing out the laws like that also makes it clear why they're not redundant in practice: the inverted forms involve double-negatives that make them incredibly hard to read. Those rules impact this PEP by way of the fact that in "LHS if COND else RHS", the "if" and "else" are actually in the same logical relation to each other as "and" and "or" are in "COND and LHS or RHS". Accordingly, if "LHS if COND else RHS" were to be reformulated as a compound instruction built from two binary instructions (akin to the way comparison chaining works) as considered in the "Risks and Concerns" section about the language level inconsistencies that the current draft introduces, then we'd expect De Morgan's laws to hold there as well: Y if X --> not ((not X) else (not Y)) X else Y --> not ((not Y) if (not X)) It hadn't occurred to me to include that observation in the PEP while updating it to switch to that base design, but it really should be there as an additional invariant that well-behaved symmetric circuit breakers should adhere to. [1] https://en.wikipedia.org/wiki/De_Morgan%27s_laws > 4) The proposa
Re: [Python-ideas] Built-in function to run coroutines
Hi Guido, On 2016-11-12 4:24 PM, Guido van Rossum wrote: I think there's a plan to add a run() function to asyncio, which would be something akin to def run(coro): return get_event_loop().run_until_complete(coro) (but perhaps with better cleanup). Please see https://github.com/python/asyncio/pull/465. Yury ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On 14 November 2016 at 12:16, Steven D'Aprano wrote: > On Mon, Nov 14, 2016 at 01:19:30AM +0100, Mikhail V wrote: > > [...] >> >> A good syntax example: >> >> >> >> a = sum (a, c) > > There is a reason why mathematicians and accountants use symbols as a > compact notation for common functions, instead of purely functional > notation. > > Here is a real-world example, the formula for compound interest: > > A = P*(1+r/100)**n > > Compare that to: > > A = mul(P, pow(sum(1, div(r, 100)), n)) > > Even Reverse Polish Notation is easier to read than that series of > functions: > > P 1 r 100 / + n ** * > > > Don't misunderstand me, functions are important, and over-use of cryptic > symbols that are only meaningful to an expert will hurt readability and > maintainability of code. But people have needed to do arithmetic for > over six thousand years, and there is no good substitute for compact > operators for basic operations. I agree. Actually I meant that both are good examples. In some cases I still find function-style better, so this: A = P*(1+r/100)**n I would tend to write it like: A = P * pow((1 + r/100) , n) For me it is slightly more readable, since ** already makes the equation inconsistent. And of course you cannot use glyphs for all possible operations, it will be total mess. Most of problems with function-style equations come from limitations of representation so for Courier font for examples the brackets are too small and makes it hard to read. With good font and rendering there no such problems. Some equation editors allow even different sized brackets - the outer extend more and more when I add nested equations, so it looks way better. > [...] >> It is kind of clear from the context, that I am speaking of syntax and >> not how things are working under the hood, or? >> If a compiler cannot optimize "a = a + 1" into an in-place operation, >> that is misfortune. > > That's not how Python works. Or at least, not without an extremely > powerful and smart compiler, like PyPy. > > In Python, integers (and floats) are immutable objects. They have to be > immutable, otherwise you would have things like this: > > x = 1 > y = x > x = x + 1 # changes the object 1 in place > print(y*10) # expect 10, but get 20 > > That's how lists work, because they are mutable: > > py> x = [1] > py> y = x > py> x[0] = x[0] + 1 > py> print(y[0]*10) # expect 1*10 = 10 > 20 > > > A "sufficiently smart" compiler can work around this, as PyPy does under > some circumstances, but you shouldn't expect this optimization to be > simple. > > [...] >> A better option would be to support unary operators so the user >> can write directly without assignment: >> >> inc (a, 1) >> >> Would mean in-place increment "a" with 1 > > That is impossible with Python's executation model, in particular the > "pass by object sharing" calling convention. I expect that this would > require some extremely big changes to the way the compiler works, > possibly a complete re-design, in order to allow pass by reference > semantics. Thanks a lot for a great explanation! So for current Python behavior, writing x = x + 1 and x += 1 Would mean the same in runtime? Namely creates a copy and assigns the value x+1 to x. Or there is still some overhead at parsing stage? Then however I see even less sense in using such a shortcut, it would be good to dismiss this notation, since it makes hard to read the code. As for Numpy, it uses anyway its own approaches, so in-place should use own syntax, e.g. like numpy.sum(x_, a) to do it in-place. Mikhail ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Built-in function to run coroutines
On 2016-11-14 1:35 PM, Sven R. Kunze wrote: What about making "run" an instance method of coroutines? That would require coroutines to be aware of the loop that is running them. Not having them aware of that is what makes the design simple and allows alternatives to asyncio. All in all I'd be strong -1 to do that. Yury ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Built-in function to run coroutines
What about making "run" an instance method of coroutines? On 14.11.2016 19:30, Yury Selivanov wrote: Hi Guido, On 2016-11-12 4:24 PM, Guido van Rossum wrote: I think there's a plan to add a run() function to asyncio, which would be something akin to def run(coro): return get_event_loop().run_until_complete(coro) (but perhaps with better cleanup). Please see https://github.com/python/asyncio/pull/465. Best, Sven ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
Currently, Numpy takes advantage of __iadd__ and friends by performing the operation in-place; there is no copying or other object created. Numpy is very thinly C, for better and worse (which is also likely where the += syntax came from). If you're doing vast amounts of numeric computation, it quickly pays off to learn a little about how C likes to store arrays in memory and that doing things like creating a whole new array of the same size and having to free up the old one for every operation isn't a great idea. I strongly dislike the notion that I'd have to use arbitrary function calls to add one to an entire array, or multiply it by 2, or add it to another array, etc. As a minor nit, np.vectorize doesn't make in-place functions, it simply makes a naive, element-wise function act as though it was vectorized. The name is unfortunate, because it does nothing to speed it up, and it usually slows it down (because of that layer). Reworking code to avoid copying large arrays by doing in-place operations is the preferred method, though not always possible. Nick On Mon, Nov 14, 2016 at 12:21 PM, Mikhail V wrote: > On 14 November 2016 at 12:16, Steven D'Aprano wrote: > > On Mon, Nov 14, 2016 at 01:19:30AM +0100, Mikhail V wrote: > > > > [...] > >> >> A good syntax example: > >> >> > >> >> a = sum (a, c) > > > > There is a reason why mathematicians and accountants use symbols as a > > compact notation for common functions, instead of purely functional > > notation. > > > > Here is a real-world example, the formula for compound interest: > > > > A = P*(1+r/100)**n > > > > Compare that to: > > > > A = mul(P, pow(sum(1, div(r, 100)), n)) > > > > Even Reverse Polish Notation is easier to read than that series of > > functions: > > > > P 1 r 100 / + n ** * > > > > > > Don't misunderstand me, functions are important, and over-use of cryptic > > symbols that are only meaningful to an expert will hurt readability and > > maintainability of code. But people have needed to do arithmetic for > > over six thousand years, and there is no good substitute for compact > > operators for basic operations. > > I agree. Actually I meant that both are good examples. > In some cases I still find function-style better, so this: > > A = P*(1+r/100)**n > > I would tend to write it like: > > A = P * pow((1 + r/100) , n) > > For me it is slightly more readable, since ** already makes the > equation inconsistent. > And of course you cannot use glyphs for all possible operations, it > will be total mess. > > Most of problems with function-style equations come from limitations > of representation > so for Courier font for examples the brackets are too small and makes > it hard to read. > With good font and rendering there no such problems. Some equation editors > allow even different sized brackets - the outer extend more and more > when I add nested equations, so it looks way better. > > > > [...] > >> It is kind of clear from the context, that I am speaking of syntax and > >> not how things are working under the hood, or? > >> If a compiler cannot optimize "a = a + 1" into an in-place operation, > >> that is misfortune. > > > > That's not how Python works. Or at least, not without an extremely > > powerful and smart compiler, like PyPy. > > > > In Python, integers (and floats) are immutable objects. They have to be > > immutable, otherwise you would have things like this: > > > > x = 1 > > y = x > > x = x + 1 # changes the object 1 in place > > print(y*10) # expect 10, but get 20 > > > > That's how lists work, because they are mutable: > > > > py> x = [1] > > py> y = x > > py> x[0] = x[0] + 1 > > py> print(y[0]*10) # expect 1*10 = 10 > > 20 > > > > > > A "sufficiently smart" compiler can work around this, as PyPy does under > > some circumstances, but you shouldn't expect this optimization to be > > simple. > > > > [...] > >> A better option would be to support unary operators so the user > >> can write directly without assignment: > >> > >> inc (a, 1) > >> > >> Would mean in-place increment "a" with 1 > > > > That is impossible with Python's executation model, in particular the > > "pass by object sharing" calling convention. I expect that this would > > require some extremely big changes to the way the compiler works, > > possibly a complete re-design, in order to allow pass by reference > > semantics. > > Thanks a lot for a great explanation! > So for current Python behavior, writing > > x = x + 1 > > and > > x += 1 > > Would mean the same in runtime? Namely creates a copy and assigns > the value x+1 to x. Or there is still some overhead at parsing stage? > Then however I see even less sense in using such a shortcut, > it would be good to dismiss this notation, since it makes hard > to read the code. > As for Numpy, it uses anyway its own approaches, so > in-place should use own syntax, e.g. like numpy.sum(x_, a) to > do it in-place. > > > Mikhail > ___ > Python-ideas mailing list > P
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On Mon, Nov 14, 2016 at 1:21 PM, Mikhail V wrote: > On 14 November 2016 at 12:16, Steven D'Aprano wrote: > > On Mon, Nov 14, 2016 at 01:19:30AM +0100, Mikhail V wrote: > > > > [...] > >> >> A good syntax example: > >> >> > >> >> a = sum (a, c) > > > > There is a reason why mathematicians and accountants use symbols as a > > compact notation for common functions, instead of purely functional > > notation. > > > > Here is a real-world example, the formula for compound interest: > > > > A = P*(1+r/100)**n > > > > Compare that to: > > > > A = mul(P, pow(sum(1, div(r, 100)), n)) > > > > Even Reverse Polish Notation is easier to read than that series of > > functions: > > > > P 1 r 100 / + n ** * > > > > > > Don't misunderstand me, functions are important, and over-use of cryptic > > symbols that are only meaningful to an expert will hurt readability and > > maintainability of code. But people have needed to do arithmetic for > > over six thousand years, and there is no good substitute for compact > > operators for basic operations. > > I agree. Actually I meant that both are good examples. > In some cases I still find function-style better, so this: > > A = P*(1+r/100)**n > > I would tend to write it like: > > A = P * pow((1 + r/100) , n) > > For me it is slightly more readable, since ** already makes the > equation inconsistent. > > Okay, then make a proposal to get the operators included as built-ins. Once you have gotten that approved, then we can start talking about which syntax is better. But the idea that people should import a module to do basic mathematical operations is a non-starter. > Most of problems with function-style equations come from limitations > of representation > so for Courier font for examples the brackets are too small and makes > it hard to read. > With good font and rendering there no such problems. Some equation editors > allow even different sized brackets - the outer extend more and more > when I add nested equations, so it looks way better. > > You shouldn't be using a variable-width font for coding anyway. That is going to cause all sorts of problems (indentation not matching up, for example). You should use a fixed-width font. But if brackets are a problem, I don't see how using more brackets is a solution (which is the case with function calls). On the contrary, I would think you would want to minimize the number of brackets. > > > [...] > >> It is kind of clear from the context, that I am speaking of syntax and > >> not how things are working under the hood, or? > >> If a compiler cannot optimize "a = a + 1" into an in-place operation, > >> that is misfortune. > > > > That's not how Python works. Or at least, not without an extremely > > powerful and smart compiler, like PyPy. > > > > In Python, integers (and floats) are immutable objects. They have to be > > immutable, otherwise you would have things like this: > > > > x = 1 > > y = x > > x = x + 1 # changes the object 1 in place > > print(y*10) # expect 10, but get 20 > > > > That's how lists work, because they are mutable: > > > > py> x = [1] > > py> y = x > > py> x[0] = x[0] + 1 > > py> print(y[0]*10) # expect 1*10 = 10 > > 20 > > > > > > A "sufficiently smart" compiler can work around this, as PyPy does under > > some circumstances, but you shouldn't expect this optimization to be > > simple. > > > > [...] > >> A better option would be to support unary operators so the user > >> can write directly without assignment: > >> > >> inc (a, 1) > >> > >> Would mean in-place increment "a" with 1 > > > > That is impossible with Python's executation model, in particular the > > "pass by object sharing" calling convention. I expect that this would > > require some extremely big changes to the way the compiler works, > > possibly a complete re-design, in order to allow pass by reference > > semantics. > > Thanks a lot for a great explanation! > So for current Python behavior, writing > > x = x + 1 > > and > > x += 1 > > Would mean the same in runtime? Namely creates a copy and assigns > the value x+1 to x. Or there is still some overhead at parsing stage? > Then however I see even less sense in using such a shortcut, > it would be good to dismiss this notation, since it makes hard > to read the code. > No, that is only the case for immutable types like floats. For mutable types like lists then x = x + y and x += y Are not the same thing. The first makes a new object, while the second does an in-place operation. Sometimes you want a new object, sometimes you don't. Having both versions allows you to control that. > As for Numpy, it uses anyway its own approaches, so > in-place should use own syntax, e.g. like numpy.sum(x_, a) to > do it in-place. > > First, there is no "_" suffix for variables that numpy could use. The syntax you are suggesting isn't possible without a major change to the basic python grammar and interpreter. Second, again, "sum(x, a)" does not add "x" and "a". Third, again, why? You st
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On 14 November 2016 at 19:57, Nick Timkovich wrote: > Currently, Numpy takes advantage of __iadd__ and friends by performing the > operation in-place; there is no copying or other object created. Numpy is > very thinly C, for better and worse (which is also likely where the += > syntax came from). If you're doing vast amounts of numeric computation, it > quickly pays off to learn a little about how C likes to store arrays in > memory and that doing things like creating a whole new array of the same > size and having to free up the old one for every operation isn't a great > idea. I strongly dislike the notion that I'd have to use arbitrary function > calls to add one to an entire array, or multiply it by 2, or add it to > another array, etc. > > As a minor nit, np.vectorize doesn't make in-place functions, it simply > makes a naive, element-wise function act as though it was vectorized. The > name is unfortunate, because it does nothing to speed it up, and it usually > slows it down (because of that layer). Reworking code to avoid copying large > arrays by doing in-place operations is the preferred method, though not > always possible. > > Nick I can understand you good. But imagine, if Numpy would allow you to simply write: A = A + 1 Which would bring you directly to same internal procedure as A += 1. So it does not currently, why? I've tested now A += 99 against A = A + 99 and there is indeed a 30% speed difference. So it functions different. Would you then still want to write += ? I never would. Also I think to implement this syntax would be almost trivial, it should just take the A = A part and do the rest as usual. And this for equally sized arrays: A = A + B Should just add B values to A values in-place. Now it gives me also ~30% speed difference compared to A += B. Mikhail ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On 11/14/2016 03:42 PM, Mikhail V wrote: On 14 November 2016 at 19:57, Nick Timkovich wrote: I can understand you good. But imagine, if Numpy would allow you to simply write: A = A + 1 Which would bring you directly to same internal procedure as A += 1. So it does not currently, why? I've tested now A += 99 against A = A + 99 and there is indeed a 30% speed difference. So it functions different. Would you then still want to write += ? I never would. Also I think to implement this syntax would be almost trivial, it should just take the A = A part and do the rest as usual. And this for equally sized arrays: A = A + B Should just add B values to A values in-place. Now it gives me also ~30% speed difference compared to A += B. If you take this file: test.py -- a = a + 1 a += 1 -- And you look at the bytecode it produces you get the following: -- $ python -m dis test.py 1 0 LOAD_NAME0 (a) 3 LOAD_CONST 0 (1) 6 BINARY_ADD 7 STORE_NAME 0 (a) 2 10 LOAD_NAME0 (a) 13 LOAD_CONST 0 (1) 16 INPLACE_ADD 17 STORE_NAME 0 (a) 20 LOAD_CONST 1 (None) 23 RETURN_VALUE -- That shows that the first and second lines are compiled _differently_. The point is that in the first line, numpy does not have the information necessary to know that "a + 1" will be assigned back to a. In the second case it does. This is presumably why they couldn't do the optimization you desire without large changes in cpython. On the second point, personally I prefer writing "a += 1" to "a = a + 1". I think it's clearer and would keep using it even if the two were equally efficient. But we are all allowed our opinions... Cheers, Thomas ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On Mon, Nov 14, 2016 at 3:42 PM, Mikhail V wrote: > On 14 November 2016 at 19:57, Nick Timkovich > wrote: > > Currently, Numpy takes advantage of __iadd__ and friends by performing > the > > operation in-place; there is no copying or other object created. Numpy is > > very thinly C, for better and worse (which is also likely where the += > > syntax came from). If you're doing vast amounts of numeric computation, > it > > quickly pays off to learn a little about how C likes to store arrays in > > memory and that doing things like creating a whole new array of the same > > size and having to free up the old one for every operation isn't a great > > idea. I strongly dislike the notion that I'd have to use arbitrary > function > > calls to add one to an entire array, or multiply it by 2, or add it to > > another array, etc. > > > > As a minor nit, np.vectorize doesn't make in-place functions, it simply > > makes a naive, element-wise function act as though it was vectorized. The > > name is unfortunate, because it does nothing to speed it up, and it > usually > > slows it down (because of that layer). Reworking code to avoid copying > large > > arrays by doing in-place operations is the preferred method, though not > > always possible. > > > > Nick > > I can understand you good. But imagine, if Numpy would allow you to > simply write: > A = A + 1 > Which would bring you directly to same internal procedure as A += 1. > So it does not currently, why? > First, because the language doesn't allow it. But more fundamentally, sometimes we don't want A = A + 1 to be the same as A += 1. Making a copy is sometimes what you want. Having both versions lets you control when you make a copy rather than being forced to always make a copy or always not. > I never would. Also I think to implement this syntax would be almost > trivial, it should > just take the A = A part and do the rest as usual. > The Python language doesn't allow it. numpy can only work with the information provided to it be the language, and the information needed to do that sort of thing is not provided to classes. Nor should it in my opinion, this is one of those fundamental operations that I think absolutely must be consistent. What you are talking about is an enormous backwards-compatibility break. It is simply not going to happen, it would break every mutable class. And you still have not provided any reason we should want to do it this way. ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On 14 November 2016 at 21:52, Todd wrote: >> I can understand you good. But imagine, if Numpy would allow you to >> simply write: >> A = A + 1 >> Which would bring you directly to same internal procedure as A += 1. >> So it does not currently, why? > > > First, because the language doesn't allow it. > > But more fundamentally, sometimes we don't want A = A + 1 to be the same as > A += 1. Making a copy is sometimes what you want. Having both versions > lets you control when you make a copy rather than being forced to always > make a copy or always not. > we don't want A = A + 1 to be the same as A += 1 " This sounds quite frustrating for me, what else could this be but A += 1? Would I want a copy with same name? How that will make sense? Making a copy is how it works now, ok. But even now there are differences, eg: A = B [1:10] is not a copy but a reference creation operation. So there are different meanings of =. > >> >> I never would. Also I think to implement this syntax would be almost >> trivial, it should >> just take the A = A part and do the rest as usual. > > > The Python language doesn't allow it. numpy can only work with the > information provided to it be the language, and the information needed to do > that sort of thing is not provided to classes. Nor should it in my opinion, > this is one of those fundamental operations that I think absolutely must be > consistent. > > What you are talking about is an enormous backwards-compatibility break. It > is simply not going to happen, it would break every mutable class. > I don't want to break anything, no. I don't have such deep knowledge of all cases, but that is very interesting how actually it must break *everything*. Say I scan through lines and find "= " and then see that on the left is an numpy array, which already indicates a unusual variable. And if look on the right side and first argument is again A, cannot I decide to make an exception and redirect it to A.__add__() or what must be there. It does already do an overloading of all these "+=" for numpy. so what will fundamentally break everything I am not sure. > And you still have not provided any reason we should want to do it this way. If let alone numpy for now and take only numeric Python variables, I find the += syntax very bad readable. I must literally stop and lean towards the monitor to decipher the line. And I am not even telling that you *cannot* do it with all possible operations: there are standard math, ORing ,XORing, conditionals, bitshifting etc. Is not this obvious? And what other reason apart from readability there can be at all? Again, how about TOOWTDI principle? some write a = a + 1, some a += 1 ? It does not add any pleasure for reading code. >> Most of problems with function-style equations come from limitations >> of representation >> so for Courier font for examples the brackets are too small and makes >> it hard to read. >> With good font and rendering there no such problems. Some equation editors >> allow even different sized brackets - the outer extend more and more >> when I add nested equations, so it looks way better. >You shouldn't be using a variable-width font for coding anyway. >That is going to cause all sorts of problems (indentation not matching up, for >example). You should use a fixed-width font. No no, I should not use monowidth fonts and nobody ever should. Most of those "indentation" problems are just though-out, All you need is IDE with good tabulation support. The readability difference between monowidth and real fonts is *huge*. Currently I am forced to use monowidth, since it is VIM's limitation and it bothers me a lot. Exception is tables with numbers, but those should not be inputed in same manner as strings, it is other medium type. Mikhail ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
This is just a matter of understanding the actual semantics of Python,
which is not the same as other languages such as C. In Python the
"assignment operator" is a way of *binding a value*. So writing:
a = some_expression
Is ALWAYS and ONLY binding the value of `some_expression` to the name `a`.
The fact that `a` might occur within `some_expression` is irrelevant to
these semantics (although it *does* mean that subsequent code won't use `a`
to refer to whatever value it used to have.
In clear and distinct contrast, "augmented assignment" is a way of calling
a method on an object. So writing:
a += some_expression
Means EXACTLY:
a.__iadd__(some_expression)
I recognize this can have the same effect if `__iadd__()` is not explicitly
defined, and when that is true, it will fall back to using `__add__()` as
the implementation. This is described in:
https://www.python.org/dev/peps/pep-0203/
To see the difference, e.g.:
>>> class Foo:
def __add__(self, other):
print("Adding", other)
return other
def __iadd__(self, other):
print("Inplace assignment", other)
return self
>>> foo = Foo()
>>> foo += 3
Inplace assignment 3
>>> foo + 3
Adding 3
3
>>> foo = foo + 3
Adding 3
>>> foo # I'm bound to an int now
3
On Mon, Nov 14, 2016 at 2:13 PM, Mikhail V wrote:
> On 14 November 2016 at 21:52, Todd wrote:
>
> >> I can understand you good. But imagine, if Numpy would allow you to
> >> simply write:
> >> A = A + 1
> >> Which would bring you directly to same internal procedure as A += 1.
> >> So it does not currently, why?
> >
> >
> > First, because the language doesn't allow it.
> >
> > But more fundamentally, sometimes we don't want A = A + 1 to be the same
> as
> > A += 1. Making a copy is sometimes what you want. Having both versions
> > lets you control when you make a copy rather than being forced to always
> > make a copy or always not.
>
> > we don't want A = A + 1 to be the same as A += 1 "
>
> This sounds quite frustrating for me, what else could this be but A += 1?
> Would I want a copy with same name? How that will make sense?
>
> Making a copy is how it works now, ok. But even now there are differences,
> eg:
> A = B [1:10]
> is not a copy but a reference creation operation. So there are
> different meanings of =.
>
> >
> >>
> >> I never would. Also I think to implement this syntax would be almost
> >> trivial, it should
> >> just take the A = A part and do the rest as usual.
> >
> >
> > The Python language doesn't allow it. numpy can only work with the
> > information provided to it be the language, and the information needed
> to do
> > that sort of thing is not provided to classes. Nor should it in my
> opinion,
> > this is one of those fundamental operations that I think absolutely must
> be
> > consistent.
> >
> > What you are talking about is an enormous backwards-compatibility
> break. It
> > is simply not going to happen, it would break every mutable class.
> >
>
> I don't want to break anything, no. I don't have such deep knowledge of all
> cases, but that is very interesting how actually it must break
> *everything*. Say I
> scan through lines and find "= " and then see that on the left is an numpy
> array, which already indicates a unusual variable.
> And if look on the right side and first argument is again A, cannot I
> decide to make an exception and redirect it to A.__add__() or what
> must be there.
> It does already do an overloading of all these "+=" for numpy.
> so what will fundamentally break everything I am not sure.
>
> > And you still have not provided any reason we should want to do it this
> way.
>
> If let alone numpy for now and take only numeric Python variables, I
> find the += syntax
> very bad readable. I must literally stop and lean towards the monitor
> to decipher
> the line. And I am not even telling that you *cannot* do it with all
> possible operations:
> there are standard math, ORing ,XORing, conditionals, bitshifting etc.
> Is not this obvious?
> And what other reason apart from readability there can be at all?
> Again, how about TOOWTDI principle? some write a = a + 1, some a += 1 ?
> It does not add any pleasure for reading code.
>
>
> >> Most of problems with function-style equations come from limitations
> >> of representation
> >> so for Courier font for examples the brackets are too small and makes
> >> it hard to read.
> >> With good font and rendering there no such problems. Some equation
> editors
> >> allow even different sized brackets - the outer extend more and more
> >> when I add nested equations, so it looks way better.
>
> >You shouldn't be using a variable-width font for coding anyway.
> >That is going to cause all sorts of problems (indentation not matching
> up, for example). You should use a fixed-width font.
>
> No no, I should not use monowidth fonts and nobody ever should. Most
> of those "indentation" problems are just though-out,
> All you need is
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On 14 November 2016 at 22:13, Mikhail V wrote: >> we don't want A = A + 1 to be the same as A += 1 " > > This sounds quite frustrating for me, what else could this be but A += 1? > Would I want a copy with same name? How that will make sense? >>> A = [1,2,3] >>> B = A >>> A = A + [1] >>> A [1, 2, 3, 1] >>> B [1, 2, 3] >>> A = [1,2,3] >>> B = A >>> A += [1] >>> A [1, 2, 3, 1] >>> B [1, 2, 3, 1] For a mutable class, A = A + something is fundamentally different from A += something. Before proposing fundamental changes to Python's semantics, please make sure that you at least understand those semantics first, and explain why your proposal is justified. There are a lot of people on this list, and the cumulative time spent reading your posts is therefore quite significant. You owe it to all those readers to ensure that your proposals are at least feasible *in the context of the Python language*. Paul ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On 15 November 2016 at 00:34, Paul Moore wrote: > On 14 November 2016 at 22:13, Mikhail V wrote: >>> we don't want A = A + 1 to be the same as A += 1 " >> >> This sounds quite frustrating for me, what else could this be but A += 1? >> Would I want a copy with same name? How that will make sense? > A = [1,2,3] B = A A = A + [1] A > [1, 2, 3, 1] B > [1, 2, 3] A = [1,2,3] B = A A += [1] A > [1, 2, 3, 1] B > [1, 2, 3, 1] > > For a mutable class, A = A + something is fundamentally different from > A += something. > > Before proposing fundamental changes to Python's semantics, please > make sure that you at least understand those semantics first, and > explain why your proposal is justified. There are a lot of people on > this list, and the cumulative time spent reading your posts is > therefore quite significant. You owe it to all those readers to ensure > that your proposals are at least feasible *in the context of the > Python language*. > > Paul Ok I am calmed down already. But how do you jump to lists already? I started an example with integers. I just want to increment an integer and I don't want to see any += in my code, it strains me. And that is exact reason I hate C syntax. Then Todd started about Numpy arrays. Ok, I just commented what I find for clearer syntax with array increment. And now lists, mutable classes... I don't use classes in my programs. I could propose something but how, if we mix everything in one big pile? Then my proposal is to make typed variables first, so I could at least consider use cases. Mikhail ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 532: A circuit breaking operator and protocol
It would be nice if coalesce(EXPR1, EXPR2, EXPR3) Evaluated the N+1 argument only if the Nth evaluated to None. Of course this may break the general patterns in the Python language. Maybe we can fake it by wrapping the expressions in lambdas: coalesce(lambda: EXPR1(), lambda EXPR2(), lambda EXPR3()) and defining a `coalesce` as def coalesce(*args): for a in args: a_val=a() if a_val is not None: return a_val return None Making a coalesce call looks painful, but allowing the called function to control the evaluation of its parameters may be useful. Suppose we can pass methods to functions; using generators to do so: Let & refer to lazy-evaluated parameters: def coalesce(&a, &b): if a is None: return b else: return a c = coalesce(expr1(), expr2()) Would be converted to : def coalesce(a, b): a = yield a_val = a() if a_val is None: b = yield b_val = b() return b_val else: return a_val exprs = [expr1, expr2] temp = coalesce() for e in exprs: try: c = temp.next(e) except StopIteration: break Or, even better... def coalesce(*&args): for a_val in args: if a_val is not None: return a_val return None c = coalesce(expr1(), expr2()) Gets converted to def coalesce(*args): for a in args: a_val = a() if a_val is not None: return a_val return None exprs = [expr1, expr2] temp = coalesce() for e in exprs: try: c = temp.next(e) except StopIteration: break ...or something like that. I can not think of other reasons for this type of expansion; maybe logical `and` can be given a magic method: "__logand__": def __logand__(self, &other): if self: return True return o c = my_object and some_other() which has a combination of immediately-evaluated parameters, and lazy-evaluated parameters: class MyClass(object): def __logand__(self): if self: yield True return other = yield return other() exprs = [some_other] temp = MyClass.__logand__(my_object) for e in exprs: try: c = temp.next(e) except StopIteration: break I hope that the acrobatics shown here might be easier to implement at a lower level; where in-line generator code collapses to simple branched logic. On 11/13/2016 1:51 AM, Nick Coghlan wrote: At that point, if we did decide to offer a builtin instead of dedicated syntax, the option I'd argue for is actually SQL's "coalesce": coalesce(EXPR1) else coalesce(EXPR2) else EXPR3 Yes, it's computer science jargon, but the operation itself is an odd one that doesn't really have an established mathematical precedent or grammatical English equivalent. Cheers, Nick. ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
>> For a mutable class, A = A + something is fundamentally different from >> A += something. >> >> Paul > Ok I am calmed down already. > But how do you jump to lists already? I started an example with integers. > I just want to increment an integer and I don't want to see any += in my code, > it strains me. And that is exact reason I hate C syntax. > Then Todd started about Numpy arrays. Ok, I just commented what I find for > clearer syntax with array increment. > And now lists, mutable classes... I don't use classes in my programs. > I could propose something but how, if we mix everything in one big pile? > Then my proposal is to make typed variables first, so I could > at least consider use cases. > Mikhail Mikhail, what Paul probably means here is that python 'operators' are actually 'syntactic sugar' for functions (it is not recommended to call these functions directly, but it is possible): e.g. if you use 'a = a + 1', python under the hood interprets it as 'a = a.__add__(1)', but if you use 'a += 1' it uses 'a.__iadd__(1)'. All python operators are implementable under functions, which is part of the spec. For integers or strings, that may not be as visible to the end user, but if you want to change the behaviour of operators, please first look at the docs [1] and try to understand them, and understand why they exists (inconclusive examples: [2). I hope that this clears up the misconceptions you may have about how python operators work. -Matthias [1]: https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types [2]: e.g. to make your library more efficient (like the operators in numpy for more efficient array operations), or adding operator functionality for numerical classes (like the decimal.Decimal class). ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
On 2016-11-15 02:59, Matthias welp wrote: [snip] e.g. if you use 'a = a + 1', python under the hood interprets it as 'a = a.__add__(1)', but if you use 'a += 1' it uses 'a.__iadd__(1)'. FTR, 'a += 1' is interpreted as 'a = a.__iadd__(1)'. ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
2016-11-14 19:40 GMT-08:00 MRAB : > On 2016-11-15 02:59, Matthias welp wrote: > [snip] > > e.g. if you use 'a = a + 1', python under the hood interprets it as >> 'a = a.__add__(1)', but if you use 'a += 1' it uses 'a.__iadd__(1)'. >> > > FTR, 'a += 1' is interpreted as 'a = a.__iadd__(1)'. Or to be even more pedantic, `a = type(a).__iadd__(a, 1)`. > > > ___ > Python-ideas mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 532: A circuit breaking operator and protocol
> > No, the conditional branching would be based on exists.__bool__ (or, > in the current working draft, is_not_none.__bool__), and that would be > "0 is not None", which would be True and hence short-circuit. > > > `__then__` is responsible for *unwrapping* the original value from the > circuit breaker when it short-circuits: it's what allows the overall > expression to return "0", even though the truth check is done based on > "0 is not None". > I see. I somehow missed that exists was a wrapping class rather than an evaluating function. --- Should you be concerned about cases where a CircuitBreaker class is used without the `else` operator? For example, if a user accidentally does something like: x = exists(input_value) or default_value If input_value is not None, the user will get an `exists` object instead of their input value. I'm worried that the distinction between `or` and `else` will not be obvious. It seems like `else` will effectively just be `or`, but with more functionality. --- I'm also still not convinced about the reasons to avoid implementing this on `or`. I'll address the points from the rationale: > defining a shared protocol for both and and or was confusing, as __then__ was the short-circuiting outcome for or , while__else__ was the short-circuiting outcome for and I wonder: Could the protocol be defined in terms of `or`, with DeMorgan's law applied behind the scenes in the case of `and`? ie: existing(x) and existing(y) => missing(y) or missing(x) > the and and or operators have a long established and stable meaning, so readers would inevitably be surprised if their meaning now became dependent on the type of the left operand. Even new users would be confused by this change due to 25+ years of teaching material that assumes the current well-known semantics for these operators With basic pass-through implementations for __then__ and __else__ attached to all classes by default, and the existing __bool__, it seems like `or` would continue to function in the same way it currently does. There are plenty of current dunder methods that are already redefined in ways that might confuse people: % on strings, set operators, etc. > Python interpreter implementations, including CPython, have taken advantage of the existing semantics of and and or when defining runtime and compile time optimisations, which would all need to be reviewed and potentially discarded if the semantics of those operations changed I can't really speak to any of this, not being familiar with the internals of any implementation. Though, it might work out that some of the code for handling `and` and `or` could be thrown out, since those operators would be transformed into conditional expressions. I very much understand the desire to not break working, optimized implementations. However, this feels a little flimsy as a reason for introducing new syntax. ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Built-in function to run coroutines
On 15 November 2016 at 04:39, Yury Selivanov wrote: > On 2016-11-14 1:35 PM, Sven R. Kunze wrote: >> >> What about making "run" an instance method of coroutines? > > That would require coroutines to be aware of the loop that is running them. > Not having them aware of that is what makes the design simple and allows > alternatives to asyncio. It also helps minimise the additional work needed for Tornado, Cython, etc to provide their own coroutine implementations. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
