RFC -- custom operators

[Steven D'Aprano]

Request for comments -- proposal to allow custom binary operators.

I'll looking for comments on custom binary operators: would it be useful, 
if so, what use-cases do you have?

The most obvious and often-requested use-case would be for a form of 
logical operator (AND, OR, XOR) that is distinct from the bitwise 
operators & | ^ but unlike the standard `and` and `or` operators, calls 
dunder methods.

The proposal is to add custom operators. A placeholder syntax might be:

spam OP eggs

which would then delegate to special dunder methods __OP__ or __ROP__ 
similar to existing operators such as + and similar.

I don't want to get into arguments about syntax, or implementation 
details, unless there is some interest in the functionality. Please focus 
on *functional requirements* only.

(1) This proposal requires operators to be legal identifiers, 
such as "XOR" or "spam", not punctuation like % and
absolutely not Unicode symbols like ∉

(2) For the sake of the functional requirements, assume that 
we can parse `spam OP eggs` without any ambiguity;

(3) This only proposes binary infix operators, not unary
prefix or postfix operators;

infix:argument1 OP argument2
prefix:   OP argument
postfix:  argument OP

(4) This does not propose to allow the precedence to be
set on a case-by-case basis. All custom operators will
have the same precedence.

(5) What should that precedence be?

(6) This does not propose to set the associativity on a
case-by-case basis. All custom operators will have
the same associativity.

(7) Should the operators be left-associative (like multiplication),
right-associative (like exponentiation), or non-associative?

# Left-associative:
a OP b OP c# like (a OP b) op c

# Right-associative:
a OP b OP c# like a OP (b op c)

In the last case, that would make chained custom operators intentionally 
ambiguous (and hence a SyntaxError) unless disambiguated with parentheses:

# Non-associative:
a OP b OP c# SyntaxError
(a OP b) OP c  # okay
a OP (b OP c)  # okay


(8) This does not propose to support short-circuiting operators.


I'm not interested in hearing theoretical arguments that every infix 
operator can be written as a function or method call. I know that. I'm 
interested in hearing about use-cases where the code is improved and made 
more expressive by using operator syntax and existing operators aren't 
sufficient.

(If there aren't any such use-cases, then there's no need for custom 
operators.)


Thoughts?



-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fishing from PyPI ?

[Chris Warrick]

On Tue, 7 Aug 2018 at 00:52, Gregory Ewing  wrote:
>
> Chris Warrick wrote:
> > The unusual domain is a common staple of Mailchimp, which is an e-mail
> > newsletter platform (it was used to mail out the announcement), and
> > they replace all links with tracking ones in their list-manage.com
> > domain.
>
> Sounds like you need to find a mail service that doesn't
> screw around with the contents of your messages. This is
> really quite obnoxious, IMO.

For the record, I’m not in any way affiliated with the PyPA. I’m just
linking to official resources that prove the legitimacy of these
e-mails.

As for finding a better host, it’s not easy. MailChimp, as annoying as
they might be, has built up a good reputation with mail hosts* (of
course, there are a bunch of other services that have such reputation
as well.) However, if you send the e-mail yourself, and big mail hosts
notice you have sent a ton of e-mail, they will probably consider you
a spammer and make your life harder. Especially if your e-mail server
is misconfigured in even the slightest way.

* https://mailchimp.com/features/email-delivery/

--
Chris Warrick 
PGP: 5EAAEA16
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC -- custom operators

[Marko Rauhamaa]

Steven D'Aprano :

> (1) This proposal requires operators to be legal identifiers, 
> such as "XOR" or "spam", not punctuation like % and
> absolutely not Unicode symbols like ∉

Oh, that's a let-down. Operator symbols get their expressive value from
visual conciseness:

   life←{↑1 ⍵∨.∧3 4=+/,¯1 0 1∘.⊖¯1 0 1∘.⌽⊂⍵}

   https://en.wikipedia.org/wiki/APL_(programming_language)#G
   ame_of_Life>

> (If there aren't any such use-cases, then there's no need for custom
> operators.)
>
> Thoughts?

I have never felt the need for custom operators in Python code. I
believe introducing them will make it harder, not easier, to read code.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC -- custom operators

[Paul Moore]

On Tue, 7 Aug 2018 at 09:03, Steven D'Aprano
 wrote:

> I'll looking for comments on custom binary operators: would it be useful,
> if so, what use-cases do you have?

I've never found a need for custom binary operators.

I can imagine some *theoretical* cases where they might be useful (but
no actual use cases!) but those would almost certainly require
relaxing one or more of the restrictions you listed, so they do not
even count as theoretical support for your suggested proposal.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RFC -- custom operators

[Rhodri James]

On 07/08/18 08:58, Steven D'Aprano wrote:

Request for comments -- proposal to allow custom binary operators.

[snip]

(1) This proposal requires operators to be legal identifiers,
 such as "XOR" or "spam", not punctuation like % and
 absolutely not Unicode symbols like ∉


Probably wise.  I'd personally be inclined to use expressive symbols 
because I, of course, am an entirely reasonable person whose every 
choice will be completely intuitive to everyone else (ho ho), but I can 
imagine other people succumbing to the temptation to trying turning 
Python in APL.


Also names are probably easier to parse.


(2) For the sake of the functional requirements, assume that
 we can parse `spam OP eggs` without any ambiguity;


Seems like a fair assumption.


(3) This only proposes binary infix operators, not unary
 prefix or postfix operators;

 infix:argument1 OP argument2
 prefix:   OP argument
 postfix:  argument OP


Other than being easier to parse, is there any particular reason for 
this choice?



(4) This does not propose to allow the precedence to be
 set on a case-by-case basis. All custom operators will
 have the same precedence.

(5) What should that precedence be?


Low.  Names, with their mandatory surrounding spaces, feel more 
separated and hence lower priority than their operands.  I would expect


  x + 1 FROBNICATE q * 4 - 2

to parse as

  (x + 1) FROBNICATE (q*4-2)


(6) This does not propose to set the associativity on a
 case-by-case basis. All custom operators will have
 the same associativity.

(7) Should the operators be left-associative (like multiplication),
 right-associative (like exponentiation), or non-associative?


Either left-associative or non-associative, probably the latter.  Most 
operators that I can think of are left-associative, but you probably 
want non-associative to avoid subtle bugs with more naturally 
right-associative operators.



(If there aren't any such use-cases, then there's no need for custom
operators.)


I can't think of any use cases I actually want off-hand.  The sorts of 
things I might apply new operators to are various bits of byte buffer 
mangling, and most of those feel more pythonic as iterative processes.


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

How do I create a variable where one index depends on the value of another index?

[giannis . dafnomilis]

Hello guys. I'm having an issue with a Python PulP MILP problem. You can find 
the simplified code that reproduces the problem here: 

from pulp import *
machines = 2
I = range(machines)
positions = 2
J = range(positions)
years = 10
T = range(years)
age = {0: 5, 1: 7}

IR = 0.06
df = 0.3

costs = {(0,0):300, (0,1):200, (1,0):500, (1,1):350}

factor = {}
finalcosts = {}
for i in I:
for j in J:
for t in T:
for k in range(age[i]):
factor[t,k] = ((1-df)**k)/((1+IR)**t)
finalcosts[i,j,t,k] = costs[i,j]*factor[t,k]

prob = LpProblem("TrialProb",LpMinimize)

Prob_vars = LpVariable.dicts("probvars", ((Machine, Position,Year, Age) for 
Machine in I for Position in J for Year in T for Age in range(age[i])),0,None, 
LpInteger)


This gives me a 'finalcosts' variable with a size of 240 which is what I want, 
with all the correct values. But the 'Prob_vars' are of a size 260, counting 
the second index k for the first index i as well. Meaning that while in 
'finalcosts' for i=0, k=0:4 and for i=2, k=0:6 (which is what I want), for the 
'Prob_vars' decision variable index k=0:6 for both i=1 & i=2.

I'm fairly new to Python so I can't quite grasp where the problem lies.

What I have tried:

I've tried all combinations that I could think of for different expressions of 
the 'Prob_vars' but nothing works properly. I also looked anywhere I could 
think of online but I can't find an answer to this.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How do I create a variable where one index depends on the value of another index?

[Peter Otten]

giannis.dafnomi...@gmail.com wrote:

> Hello guys. I'm having an issue with a Python PulP MILP problem. You can
> find the simplified code that reproduces the problem here:
> 
> from pulp import *
> machines = 2
> I = range(machines)
> positions = 2
> J = range(positions)
> years = 10
> T = range(years)
> age = {0: 5, 1: 7}
> 
> IR = 0.06
> df = 0.3
> 
> costs = {(0,0):300, (0,1):200, (1,0):500, (1,1):350}
> 
> factor = {}
> finalcosts = {}
> for i in I:
> for j in J:
> for t in T:
> for k in range(age[i]):
> factor[t,k] = ((1-df)**k)/((1+IR)**t)
> finalcosts[i,j,t,k] = costs[i,j]*factor[t,k]
> 
> prob = LpProblem("TrialProb",LpMinimize)
> 
> Prob_vars = LpVariable.dicts("probvars", ((Machine, Position,Year, Age)
> for Machine in I for Position in J for Year in T for Age in
> range(age[i])),0,None, LpInteger)

That should probably be age[Machine], not age[i] in the line above.

> This gives me a 'finalcosts' variable with a size of 240 which is what I
> want, with all the correct values. But the 'Prob_vars' are of a size 260,
> counting the second index k for the first index i as well. Meaning that
> while in 'finalcosts' for i=0, k=0:4 and for i=2, k=0:6 (which is what I
> want), for the 'Prob_vars' decision variable index k=0:6 for both i=1 &
> i=2.
> 
> I'm fairly new to Python so I can't quite grasp where the problem lies.
> 
> What I have tried:
> 
> I've tried all combinations that I could think of for different
> expressions of the 'Prob_vars' but nothing works properly. I also looked
> anywhere I could think of online but I can't find an answer to this.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How do I create a variable where one index depends on the value of another index?

[giannis . dafnomilis]

Hey Peter.

This worked like a charm! I can't believe I did not think of that, after 
wasting so many hours on it.

Thank you so much for the help!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How do I create a variable where one index depends on the value of another index?

[Peter Otten]

giannis.dafnomi...@gmail.com wrote:

> Hey Peter.
> 
> This worked like a charm! I can't believe I did not think of that, after
> wasting so many hours on it.
> 
> Thank you so much for the help!

You can make these things easier to debug by breaking your code into small 
functions. If there were separate functions

def make_finalcosts(...):
   ...  # i is a local variable here

def make_probvars(...):
   ...  # i is not defined here and there's no global i

you'd have seen a NameError exception similar to the one below:

>>> def example(m, n):
... return [i + k for j in range(m) for k in range(n)]
... 
>>> example(1, 2)
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2, in example
  File "", line 2, in 
NameError: name 'i' is not defined

Writing a function in such a way that the result only depends on the 
function's arguments allows for easy testing.

-- 
https://mail.python.org/mailman/listinfo/python-list

How to make python pick up my new-and-shiny openssl shared object

[Fetchinson . via Python-list]

The highest version of openssl available on my system is 1.0.0 which
is not good enough for pip these days (or github for that matter). So
I've installed 1.1.0 to a custom location /home/fetch/opt. But if I do

import ssl
ssl.OPENSSL_VERSION

it still shows me that it is using the system default 1.0.0. How do I
tell python to use /home/fetch/opt for the ssl module? Note that I
have /home/fetch/opt as the first entry in LD_LIBRARY_PATH. Also, I
know for a fact that I don't need to recompile python for this so
please don't suggest "just recompile python with the new openssl
library" as the solution :)

By the way my python is 2.7.3.

-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make python pick up my new-and-shiny openssl shared object

[Christian Heimes]

On 2018-08-08 00:07, Fetchinson . via Python-list wrote:
> The highest version of openssl available on my system is 1.0.0 which
> is not good enough for pip these days (or github for that matter). So
> I've installed 1.1.0 to a custom location /home/fetch/opt. But if I do
> 
> import ssl
> ssl.OPENSSL_VERSION
> 
> it still shows me that it is using the system default 1.0.0. How do I
> tell python to use /home/fetch/opt for the ssl module? Note that I
> have /home/fetch/opt as the first entry in LD_LIBRARY_PATH. Also, I
> know for a fact that I don't need to recompile python for this so
> please don't suggest "just recompile python with the new openssl
> library" as the solution :)

Hi,

first of all, you need to use the library directory for LD_LIBRARY_PATH.
It's the directory that contains libssl*.so, probably
/home/fetch/opt/lib or /home/fetch/opt/lib64.

You may also have to recompile Python yourself. OpenSSL 1.0.2 is not
ABI-compatible with OpenSSL 1.0.0. In case you want to use OpenSSL
1.1.0, you must update to a more recent version of Python, too. OpenSSL
1.1.0 support was added in 2.7.13.

Christian

-- 
https://mail.python.org/mailman/listinfo/python-list