Re: A question on modification of a list via a function invocation

2017-09-03 Thread Stephan Houben
Op 2017-08-17, Rustom Mody schreef :
> On Thursday, August 17, 2017 at 6:49:19 AM UTC+5:30, Mok-Kong Shen wrote:
>> Am 17.08.2017 um 02:41 schrieb Steve D'Aprano:
>> > By reference and by value are not the only two conventions.
>> > 
>> > Perhaps if you go back to the 1950s you might be able to argue that
>> > "reference" and "value" are the only two conventions, but
>> > alternatives have existed for many decades, since *at least* 1960
>> > when Algol introduced "call by name".

I'm a bit late to this discussion, but pelase allow me to add some (to
me at least) fascinating history to this.

In 1966, a very influential paper was published:
P.J. Landin, "The Next 700 Programming Languages"
See: https://www.cs.cmu.edu/~crary/819-f09/Landin66.pdf

In this paper, Landin decribes a "family of unimplemented computing
languages that is intended to span differences of application area by a
unified framework". He calls this language (or family of languages)
ISWIM. It is a language with Lisp-like semantics but "Algol-like"
(i.e. infix) syntax, dynamically typed, and he introduces the novel
idea to have "Indentation, used to indicate program structure." 

Sounds familiar to anybody? Yep, this is basically proto-Python.

Anyway, then there is a later paper (from 1974) by
G.D. Plotkin, "Call-by-name, call-by-value and the λ-calculus"
(see http://www.sciencedirect.com/science/article/pii/0304397575900171 ).

In this paper, Plotkin "examines the old question of the relationship
between ISWIM and the λ-calculus, using the distinction between
call-by-value and call-by-name." Yep, in 1974, what to call the calling
convention of proto-Python was already an "old question".

In this paper, Plotkin introduces the λV-calculus, the call-by-value
lambda-calculus, to formalize what it is what ISWIM (and Python) are
actually doing. This paper is, to the best of my knowledge, the closest
thing to an "official" definition of what call-by-value actually means.

Needless to say, according to the definition in Plotkin's paper, Python
is "call-by-value".

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


asyncio.gather cancellation behaviour

2017-09-08 Thread Stephan Houben
Hi all,

I am a bit mystified about the rationale of the cancellation
behaviour of asyncio.gather. 

  Case 1:
  "If the outer Future is cancelled, all children (that have not completed
  yet) are also cancelled."

  Case 2:
  "If any child is cancelled, this is treated as if it raised
  CancelledError – the outer Future is not cancelled in this case. 
  (THIS IS TO PREVENT THE CANCELLATION OF ONE CHILD TO CAUSE OTHER CHILDREN TO
  BE CANCELLED.)" [capitalization mine]

Indeed I can observe this behavior. However, I would like to further
understand the reasoning for not cancelling in case 2.

Outside asyncio.gather, cancelling an "outer future" does NOT cancel
the "inner future" on which the outer future is currently await-ing, while
cancelling the "inner future" will raise a CancelledError which will
cancel the outer future.

Example:
   
import asyncio

f1 = asyncio.Future()

async def make_f2():
await f1

f2 = asyncio.ensure_future(make_f2())

f2.cancel() # cancelling f1 instead will cancel BOTH f1 and f2

loop = asyncio.get_event_loop()
try:
loop.run_until_complete(f2)
except asyncio.CancelledError:
print("got CancelledError")

print("f1 cancelled: ", f1.cancelled()) # prints False
print("f2 cancelled: ", f2.cancelled()) # prints True

So cancellation "normally" proceeds from inner future -> outer future.

It seems somebody worked hard to reverse the direction in case of
asyncio.gather. Why?

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Run Windows commands from Python console

2017-09-08 Thread Stephan Houben
Op 2017-09-06, Rick Johnson schreef :

> One of the nice (current) features of Tkinter menus (that i
> sometimes miss on my windows box!) is the ability to "tear-
> off" a menu cascade and use it as a sort of "pseudo tool
> bar". 

I was under the impression that Tk also supported tear-off
menus under Windows (but not under macOS).
However, many applications apparently explicitly suppress 
this functionality by doing

  Menu(..., tearoff=0)

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Best way to insert sorted in a list

2017-09-08 Thread Stephan Houben
Op 2017-09-08, logonve...@gmail.com schreef :
> On Saturday, June 18, 2011 at 2:23:10 AM UTC+5:30, SherjilOzair wrote:
>> There are basically two ways to go about this.
>> One is, to append the new value, and then sort the list.
>> Another is to traverse the list, and insert the new value at the
>> appropriate position.
>> 
>> The second one's complexity is O(N), while the first one's is O(N *
>> log N).
>> 
>> Still, the second one works much better, because C code is being used
>> instead of pythons.

Python uses the Timsort ( https://en.wikipedia.org/wiki/Timsort )
algorithm. Timsort is O(N) in the special case of a list of N elements
where the first N-1 are already sorted and the last one is arbitrary.

So appending the value and then calling sort() is in fact O(N) in Python
(hence asymptotically optimal), and also practically fast since the
sort() is implemented in C.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Using Python 2

2017-09-08 Thread Stephan Houben
Op 2017-09-08, Stefan Ram schreef :
>   OTOH, there are those killjoys who complain about
>   "too many parentheses" in programs written in the
>   "simple language".

Which is clearly nonsense since it is easy to show that any working
program in said simple language contains *precisely enough* parentheses.

;-)

(Fortunate that I commented out the unbalanced closing parenthesis.)

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: The Incredible Growth of Python (stackoverflow.blog)

2017-09-10 Thread Stephan Houben
Op 2017-09-10, Chris Angelico schreef :
> Want to make something iterable? Define __iter__. Want to make it
> async-iterable (with "async for")? Define __aiter__. It's a bit clunky
> if you want the same object to be iterable both ways, but I don't know
> of any real-world situations where that's the case.

Would we not eventually want a file object to deliver its lines
asynchronously (with non-blocking reads under the hood) if
iterated over with "async for", while preserving the current
blocking behavior in the "for" case?

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: People choosing Python 3

2017-09-10 Thread Stephan Houben
Op 2017-09-10, Marko Rauhamaa schreef :
> As an application developer, I can't make the customers depend on EPEL.
> It's Python2 until the distro comes with Python3.

Why not bundle the Python interpreter with your application?
It seems to work for Windows developers...

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: People choosing Python 3

2017-09-11 Thread Stephan Houben
Op 2017-09-10, Marko Rauhamaa schreef :
> Stephan Houben :
>>
>> Why not bundle the Python interpreter with your application?
>> It seems to work for Windows developers...
>
> I've seen that done for Python and other technologies. It is an
> expensive route to take. Also, it can be insecure. When vulnerabilities
> are found, they are communicated to the maintainers of, say, Python.
> When Python is fixed and released, the vulnerability is revealed, but
> the version bundled with your product is still broken. You have to be
> prepared perform an emergency release of your product and hope you don't
> mess things up.

To each his own, but this is not different from any other
third-party package your application depends on.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Run Windows commands from Python console

2017-09-11 Thread Stephan Houben
Op 2017-09-10, Rick Johnson schreef :
> It seems to me the best solution is for the TCL/Tk folks to
> provide a configuration utility that stores user preferences
> in the registry, or some other OS provided mechanism, as to
> have these settings reset on every invocation of the
> application would be inefficient from an enduser
> perspective. 

You mean, like the existing .Xdefaults mechanism
(yes Tk also supports that on Windows),
and the "option" mechanism?

https://www.tcl.tk/man/tcl8.6/TkCmd/option.htm

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: The Incredible Growth of Python (stackoverflow.blog)

2017-09-11 Thread Stephan Houben
Op 2017-09-10, Marko Rauhamaa schreef :
> Stephan Houben :
>
>> Would we not eventually want a file object to deliver its lines
>> asynchronously (with non-blocking reads under the hood) if
>> iterated over with "async for", while preserving the current
>> blocking behavior in the "for" case?
>
> I'm not exactly sure what your point is.

I mean that I would imagine that 

1. functionality as is today available in `aiofiles' would
   at some point be integrated into the standard library,
   and

2. that this might be done in such a way that there is
   no distinction anymore between a normal file object
   and an "aiofiles" file object, unlike today.

> As for file objects supporting asynchronous iterators, I agree they
> should. 

OK, that is essentially my point 2, above.

> Linux is not quite ready for nonblocking file access yet (the
> kernel developers are busy trying to make it happen).
>
> Note that you will not only need an async version of a file iterator but
> also versions for the "open()" function, directory walking etc.

open() already supports non-blocking mode (as does read() and write(), of
course).
readdir() is indeed currently always blocking.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: mutiprocessing gui

2017-09-11 Thread Stephan Houben
Op 2017-09-11, Antoon Pardon schreef :
> When one wants to combine multithreading and gui programming
> all sorts of issues arise. So I wonder how one might combine
> multiprocessing with gui programming.
>
> gui libraries typically have some registration mechanisme,
> where you can register a call back for when data is available
> on a network connection. Could one write a wrapper so that
> the multiprocessing.Queue or multiprocessing.Pipe could be
> usable like that?

Are you aware of Quamash?
This allows you to combine a Qt GUI with asyncio-style code for,
say, handing network connections.

https://github.com/harvimt/quamash

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-12 Thread Stephan Houben
Op 2017-09-12, gerlando.fala...@gmail.com schreef :

> Notice however how I'd have to create those Arrays dynamically in the
> producer thread. Would I then be able to pass them to the consumer by
> putting a reference in a queue? 

Yes.

> I wouldn't want them to be pickled at all in that case, of course.

Essentially only an address pointing into a piece of shared memory is
pickled and transmitted then.

> Also, why do you say it only works on Unix? I couldn't find any
> reference to such limitation in the documentation.

There is no such limitation.
It's supposed to work on all platforms (including Windows).

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-12 Thread Stephan Houben
Op 2017-09-12, Thomas Jollans schreef :
> I'm not sure actually. Maybe someone else here can help. I have a hunch
> that on Windows the memory might not be shared in the same way that it
> would on Linux/Unix, since Windows simply doesn't have the same process
> forking capabilities as Unix. 

`multiprocessing` is not relying on the POSIX "inherit an anonymous
mmap()" semantics to get a piece of memory shared across processes.

1. On Windows, it uses the Windows-specific functionality to
   associate a tagname with an mmap and open the specific mmap by tagname.

2. On Posix platforms, it creates a file which is opened and immediately
   unlink()-ed, and the file descriptor is communicated to the child
   process using a UNIX-domain socket.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Fw: Problems Installing Python36

2017-09-12 Thread Stephan Houben
Op 2017-09-12, Thomas Jollans schreef :
> This isn't the first time I've someone with this issue here. It's
> probably putting off plenty of potential new users who don't make as
> much effort to find a solution. I can't say I understand the ins and
> outs of installing things on Windows... is there anything that can be done?

I just added issue 31427 on the Python bug tracker proposing
adding this to the Windows FAQ.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: The Incredible Growth of Python (stackoverflow.blog)

2017-09-12 Thread Stephan Houben
Op 2017-09-12, Tim Golden schreef :

> I agree. Except for the unusual case where someone's mistakenly chosen 
> to use, eg, Python 2.4 because they're using an old text book which 
> mentions it as the current version, most people are using the version 
> which suits them for one reason or another.

If it is not clear from the original post that they are consciously
using an old version, I will often try to politely suggest they use the
latest version (i.e. "If there is no particular reason for you to use
Python 1.5.2, may I suggest that you use 3.6 instead?").

The reason is that:

* On many operating systems, just typing "python" will give you
  Python2.7 at best (e.g. macOS!). And the OP may not be aware that
  there are more recent versions.

* In businesses, it is especially common to run on some RHEL version
  $ANCIENT; not so long ago I had a machine where typing "python"
  presented me with Python 2.3!

I agree that *badgering* them about it (as opposed to suggesting it
*once*) is a bad idea.

> And, if I may put my 2.5p-worth in here, they're probably using the 
> operating system which suits them. (Maybe because their employer has 
> said so, or because they're familiar or whatever). So saying, as people 
> occasionally do, "Upgrade to Linux", even with half-a-wink, is not 
> really that funny or helpful.

It's not really the same, though.

Changing the OS is a big undertaking and affects all their programs,
just installing version 3.6 of Python as a private user shouldn't affect
anything else. You can install python in your home directory on your
ancient RHEL box and leave the system Python happily at 2.3.

I nowadays typically never bother with the system Python for my own
development and just install a recent version of my choice locally.  It
saves a lot of headaches. The system Python is there to run system
programs.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python dress

2017-09-12 Thread Stephan Houben
Op 2017-09-12, Jona Azizaj schreef :
> It looks very nice, thanks for sharing :)

   print(insertionSort)

It's even Python3-compliant!

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Research paper "Energy Efficiency across Programming Languages: How does energy, time, and memory relate?"

2017-09-19 Thread Stephan Houben
Op 2017-09-19, Steven D'Aprano schreef :

> There is a significant chunk of the Python community for whom "just pip 
> install it" is not easy, legal or even possible. For them, if its not in 
> the standard library, it might as well not even exist.

But numpy *is* in the standard library, provided you download the
correct version of Python, namely the one from:

https://python-xy.github.io/

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Fw: Problems Installing Python36

2017-09-22 Thread Stephan Houben
Op 2017-09-20, Irmen de Jong schreef :
> The only thing I can think of is that it asks windows update to
> install said KB update but that it depends on something else that
> isn't installed or that the user running the installation doesn't have
> the rights to install windows updates. (I suspect something will be
> logged in the event viewer somewhere...?)

Yeah, I suppose something like that. Presumably you can lock down
Windows in some way that it won't install the KB, but that is clearly
not the factory default. 

> Or that it doesn't attempt to download it at all and that we are
> misinformed :P

:-(

> Btw, I personally never had any issues installing Python on Windows.

I was vaguely tempted to offer the Mingw-w64 (GCC) Python as an
alternative, since it doesn't rely on any optionally-installed Microsoft
DLLs and so avoids this issue. But I suppose that is not really the
newbie-friendly solution the OP was looking for...

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Even Older Man Yells at Whippersnappers

2017-09-22 Thread Stephan Houben
Op 2017-09-21, Thomas Jollans schreef :
> On 2017-09-19 20:21, Stefan Ram wrote:
>> I do not use UTF-8
>> 
>
> Why on earth not?!

Even *More* Older Man Yells at UTF-8?
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Assertions

2017-09-22 Thread Stephan Houben
Op 2017-09-22, Thomas Jollans schreef :

> Just to make the implication explicit:
>
 from math import nan
 nan is nan
> True
 nan == nan
> False
 nan != nan
> True


To add to the fun:

>>> nan is nan
True

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Fw: Problems Installing Python36

2017-09-22 Thread Stephan Houben
Op 2017-09-22, Irmen de Jong schreef :
> On 09/22/2017 08:34 PM, Stephan Houben wrote:
>
>> I was vaguely tempted to offer the Mingw-w64 (GCC) Python as an
>> alternative, since it doesn't rely on any optionally-installed Microsoft
>> DLLs and so avoids this issue. But I suppose that is not really the
>> newbie-friendly solution the OP was looking for...
>
> Mingw? Perhaps better to choose Anaconda/Miniconda instead if you go for
> an alternative implementation...

Anaconda is still based on MSVC and so still relies on the MSVC C library.

If installing the MSVC C linrary is the problem, then I don't think
Anaconda solves that.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: How to share class relationship representations?

2017-09-22 Thread Stephan Houben
Op 2017-09-22, Pavol Lisy schreef :
> On 9/19/17, leam hall  wrote:
>> I'm working on designing the classes, sub-classes, and relationships in my
>> code. What is a good visual way to represent it so it can be stored in git
>> and shared on the list without large images or attachments?
>>
>> Thanks!
>>
>> Leam
>
> https://stackoverflow.com/questions/29586520/can-one-get-hierarchical-graphs-from-networkx-with-python-3#29597209

For direct inclusion in source code, what about plain text with 
Unicode box drawing characters?

┏━━┓
┃object┃
┗━━━┳━━┛
┃
 ┏━━┻━━┓
 ┃float┃
 ┗━┛

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: [Tutor] beginning to code

2017-09-23 Thread Stephan Houben
Op 2017-09-23, Rick Johnson schreef :
> These pissing contests over how values are passed in Python
> are totally irrelevant. What does it matter? Nothing will be
> gained or lost by arguing over which is true, or not. Unless
> the distinction is preventing you from doing something that
> you'd like to do, or unless you want to argue that one
> "value passing method" would bring X, Y or Z benefits over
> the other, what does it matter?

Amen.

For what it's worth, this discussion lead me to do a bit of historical
research into how these terminologies came to be (also to see if there
is some sort of "official" definition of what they actually mean).

The earliest to which I can trace the discussion is on the context of
the definition of the Algol programming language. Algol had both
"call by value" and "call by name".

Then the whole "call by XXX" terminology took off and people started
talking about "call by copy in/out" and whatever. The "call by
reference" terminology was introduced to describe what Fortran had been
doing all along.

The CLU people (primary Barbara Liskov) introduced "call by object
sharing", to describe what CLU did. This matches pretty well what Python
does, and what Java does, and what basically every programming language
invented in the last 25 years does.

Now, as I said, what Java and what CLU do is pretty similar, but in
the Java community this very same thing is called "call by value".
As far as I can follow, this is for the following reason:

 * The Algol report invented "call by value" and "call by name" and was
   very influential.

 * In particular, the Report on the Scheme language was heavily
   influenced by the Algol report. In the Scheme report, Scheme is
   described as being "call by value", again probably because of
   influence of the Algol report, and Scheme is definitely NOT
   "call by name". Note that in our terminology, Scheme should
   properly be called "call by object [sharing]".

 * Guy Steele, who was involved in the Scheme standard, then went 
   on to work on the Java language definition.

So Java got its terminology from the Algol->Scheme->Java route.
Python got it from CLU.

As an aside, in the academic literature, "call by value" is almost
always contrasted with "call by name" (nobody seems to have ever
published a paper discussing "call by reference").
Typically, this comparison is done in calculi which even lack
assignment so that the difference between call by value and call by
reference would be unobservable anyway.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Calling methods without objects?

2017-09-26 Thread Stephan Houben
Op 2017-09-25, Stefan Ram schreef :

>   So, is there some mechanism in Python that can bind a method
>   to an object so that the caller does not have to specify the
>   object in the call?
>
>   If so, how is this mechanism called?
>

Others have already explained the details how functions become bound
methods, but I would just point out that it is an instance of a piece of
very general functionality: the descriptor protocol.

https://docs.python.org/3.6/howto/descriptor.html

With this, you can create your own objects which do some arbitrary
special thing when accesses on an instance.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Parentheses (as after "print")

2017-09-26 Thread Stephan Houben
Op 2017-09-26, Stefan Ram schreef :
>   What happened? I woke up today in parens mood. So I typed:
>
> import( operator )
>
>   Python told me that I should type:
>
> import operator
>
>   . Fine, Python conditioned me to omit the parens. 
>   So now I was in noparens mood. So I typed:
>
> help operator
>
>   . Oops!

The explanation is of course that `import` is a syntactic keyword
and `help` is an ordinary function.

But if you want to be lazy, you can use the `ipython` interpreter
(https://ipython.org) with the %autocall feature...

  $ ipython
  Python 3.6.2 (default, Aug  3 2017, 16:08:52)
  Type 'copyright', 'credits' or 'license' for more information
  IPython 6.2.0 -- An enhanced Interactive Python. Type '?' for help.

  In [1]: %autocall
  Automatic calling is: Smart

  In [2]: help help
  Help on _Helper in module _sitebuiltins object:

  class _Helper(builtins.object)


Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: merits of Lisp vs Python

2017-09-30 Thread Stephan Houben
Op 2017-09-27, Robert L. schreef :
> (sequence-fold + 0 #(2 3 4))
> ===>
> 9
>
> In Python?

>>> sum([2, 3, 4])
9
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: LOOP with fixed final index value

2017-09-30 Thread Stephan Houben
Op 2017-09-27, Robert L. schreef :
>> > (defun myloop (initial final increment)
>> >   (loop for i = initial then (+ i increment)
>> > while (< i final)
>> > do (print i)
>> > finally (let ((i final)) (print i
>> >

> In Python?

myloop = lambda *args: print("{}{}".format("".join(map("{}\n".format,
  range(*args))), args[1]))

Of course.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: merits of Lisp vs Python

2017-09-30 Thread Stephan Houben
Op 2017-09-30, Marko Rauhamaa schreef :
> Robert L. is only trolling. He uses fake technical comments to spread
> white supremacy in his signatures.

My apologies.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: newb question about @property

2017-10-01 Thread Stephan Houben
Op 2017-10-01, Bill schreef :
> I watched an example on YouTube where someone wrote a simple descriptor 
> ("@Time_it) to output the amount of time that it took ordinary functions 
> to complete.To be honest, I AM interested in descriptors. 

Are you sure you are not confusing deSCRIPTtors and deCORAtors here?

@Time_it 

is decorator syntax.

Despite the similarity in the words, there are totally different things.

Descriptors are objects with __get__, and optionally __set__ and
__delete__ methods (i.e. they implement the descriptor protocols).

Decorators aren't really an official type, but loosely speaking these
are any functions which can be applied meaningfully with a single
function or class as argument. Some very mundane functions can be
(ab)used as decorators.

In [1]: @repr
   ...: def hello():
   ...: pass
   ...:

In [2]: hello
Out[2]: ''

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: newb question about @property

2017-10-01 Thread Stephan Houben
Op 2017-10-01, Bill schreef :
> Steve D'Aprano wrote:
>>
>> [1] Technically, the interpreter knows nothing about properties. What
>> it cares about is *descriptors*. Properties are just one kind of
>> descriptor, as are methods. But I'm intentionally not talking about
>> the gory details of descriptors. Feel free to ask if you care, but
>> honestly, you don't need to care unless you are writing your own
>> descriptor class.
>>
> I found the following page to be a very easily accessible discussion 
> about descriptors (it represents the state of my knowledge about 
> descriptors).   You got more?  : )
>
> https://www.programiz.com/python-programming/decorator

I found the page to be a discussion about decorators
(unsurprisingly given the URL) and not containing
the word "descriptor" at all...

Note that the remark from Steve is on the topic of descriptors.
I suppose the first advice to anybody wanting to learn about
either descriptors or decorators is to not confuse them
with the other thing.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: How to determine lowest version of Python 3 to run?

2017-10-04 Thread Stephan Houben
Op 2017-10-05, Ben Finney schreef :
> Christopher Reimer  writes:
>
>> How do I determine the lowest version of Python to [declare that my
>> code supports]?
>
> You can determine that by installing all the Python versions you want to
> try, and running your code's unit test suite on each of them.

An easy way to get a bunch of different Python versions is 
to just pull the various Docker containers for them.

https://hub.docker.com/r/_/python/

Then run your unit tests in each of them.

Note that Python 3.3 support officially ended on 2017-09-29,
so at the moment I wouldn't bother to support anything lower than
3.4. That means testing 3.4, 3.5 and 3.6 (and 3.7 prerelease if
you want to be thorough).

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Multithreaded compression/decompression library with python bindings?

2017-10-04 Thread Stephan Houben
Op 2017-10-04, Paul Moore schreef :
> On 4 October 2017 at 16:08, Steve D'Aprano  wrote:
>> On Wed, 4 Oct 2017 08:19 pm, Thomas Nyberg wrote:
>>
>>> Hello,
>>>
>>> I was wondering if anyone here knew of any python libraries with
>>> interfaces similar to the bzip2 module which is also multithreaded in
>>> (de)compression? Something along the lines of (say) the pbip2 program
>>> but with bindings for python?

In a pinch:

  with open("myoutfile.gz", "wb") as f:
  sp = subprocess.Popen(("gzip",), stdin=subprocess.PIPE, stdout=f)
  sp.stdin.write(b"Hello, world\n")
  sp.stdin.close()

Does compression in a separate process ;-)

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Multithreaded compression/decompression library with python bindings?

2017-10-05 Thread Stephan Houben
Op 2017-10-05, Thomas Nyberg schreef :
> Btw if anyone knows a better way to handle this sort of thing, I'm all
> ears. Given my current implementation I could use any compression that
> works with stdin/stdout as long as I could sort out the waiting on the
> subprocess. In fact, bzip2 is probably more than I need...I've half used
> it out of habit rather than anything else.

lzma ("xv" format) compression is generally both better and faster than
bzip2. So that gives you already some advantage.

Moreover, the Python lzma docs say:

"When opening a file for reading, the input file may be the concatenation
of multiple separate compressed streams. These are transparently decoded
as a single logical stream."

This seems to open the possibility to simply divide your input into,
say, 100 MB blocks, compress each of them in a separate thread/process
and then concatenate them. 

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: How to determine lowest version of Python 3 to run?

2017-10-06 Thread Stephan Houben
Op 2017-10-06, Christopher Reimer schreef :

> So I got tox and tox-docker installed. When I went to install Docker
> for Windows, it wouldn't work because Hyper-V wasn't available on
> Windows 10 Home. After paying Microsoft $99 for the privilege, I got
> Windows 10 Pro installed and Docker started working. I've read that
> all the cool programming kids are using Docker these days. I certainly
> hope it was worth the cost. O_o

You could have just used a Linux VM in Virtualbox for $0, and run Docker
in that.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Installing tkinter on FreeBSD

2017-10-24 Thread Stephan Houben
Op 2017-10-23, Thomas Jollans schreef :
> On 24/10/17 00:16, Dick Holmes wrote:
>> I am trying to use tkinter on a FreeBSD system but the installed 
>> versions of Python (2.7 and 3.6) don't have thinter configured. I tried 
>> to download the source (no binaries available for FreeBSD).

What version of FreeBSD is that? 
On 11.1 I get:

$ pkg search tkinter
py27-tkinter-2.7.14_6  Python bindings to the Tk widget set (Python 2.7)
py34-tkinter-3.4.7_6   Python bindings to the Tk widget set (Python 3.4)
py35-tkinter-3.5.4_6   Python bindings to the Tk widget set (Python 3.5)
py36-tkinter-3.6.2_6   Python bindings to the Tk widget set (Python 3.6)
pypy-tkinter-5.8.0 PyPy bindings to the Tk widget set

and for sure installing py36-tkinter-3.6.2_6 works fine.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Windows alternative: multiprocessing.connection.wait on Pipe, Tkinter File Handlers

2017-10-26 Thread Stephan Houben
Op 2017-10-23, Thomas Jollans schreef :
> You might wait in a thread
> and somehow (no idea what the best way to do this is in tkinter) pass a
> message to the GUI thread when it's done.

AFAIK, this is a real problem in Tkinter on Windows.
On Posix you can use the self-pipe trick.
But on Windows, Tkinter cannot wait on a pipe.

I see two solutions (well, three, really):

1. Use a different toolkit, e.g. PyQt has explicit support for notifying
   the GUI thread from another thread.

2. Use Cygwin-based Python. If this is an option, the Cygwin people did 
   already all the heavy lifting for providing Posix-like select()
   semantics.

3. Regular polling. This has been rejected by the OP, but in my
   experience can produce reasonable results when done "properly",
   i.e. use the Tkinter "after" method with a reasonable time interval
   (in the past I have used a strategy of starting with 10 ms and then,
   on no event, slowly back off to polling every 200ms).

   It is by far the simplest solution, and AFAIK the only one which will
   work with the standard Python distribution + Tkinter.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: doubling the number of tests, but not taking twice as long

2018-07-16 Thread Stephan Houben
Op 2018-07-16, Larry Martell schreef :
> I had some code that did this:
>
> meas_regex = '_M\d+_'
> meas_re = re.compile(meas_regex)
>
> if meas_re.search(filename):
> stuff1()
> else:
> stuff2()
>
> I then had to change it to this:
>
> if meas_re.search(filename):
> if 'MeasDisplay' in filename:
> stuff1a()
> else:
> stuff1()
> else:
> if 'PatternFov' in filename:
> stuff2a()
>else:
> stuff2()
>
> This code needs to process many tens of 1000's of files, and it runs
> often, so it needs to run very fast. Needless to say, my change has
> made it take 2x as long. 

It's not at all obvious to me.  Did you actually measure it?
Seems to depend strongly on what stuff1a and stuff2a are doing.

> Can anyone see a way to improve that?

Use multiprocessing.Pool to exploit multiple CPUs?

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Can pip install packages for all users (on a Linux system)?

2018-07-25 Thread Stephan Houben
Op 2018-07-24, John Ladasky schreef :
> I believe that I now have tensorflow 1.8 installed twice on my system,
> once for each user.  If anyone can share how to convince pip to behave
> like Synaptic, I would appreciate it.  Thanks.

I would recommend against using pip to install packages into the system
Python. The reason is that you may get into a conflict with the package
manager. You can try to be careful and not install tensorflow using
Synaptic, but once you install some other package which happens to
depend on tensorflow, you will start getting package manager conflicts.

An alternative is to install Python yourself (from source, without the package
manager) in, say, /opt/python. You are then in splendid isolation from
the package manager and can install any version of Python and tensorflow
you desire.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Checking whether type is None

2018-07-25 Thread Stephan Houben
Op 2018-07-24, Chris Angelico schreef :
> On Wed, Jul 25, 2018 at 9:18 AM, Rob Gaddi
> wrote:
>> On 07/24/2018 01:07 PM, Chris Angelico wrote:
>> I suppose one valid usage would be this sort of thing:
>>
>> fn = {
>> int: dispatchInt,
>> str: dispatchStr,
>> list: dispatchList,
>> type(None): dispatchNone
>> }[type(x)]
>> fn(x)
>>
>
> True, but that would be useful only in a very few situations, where
> you guarantee that you'll never get any subclasses. So if you're
> walking something that was decoded from JSON, and you know for certain
> that you'll only ever get those types (add float to the list and it's
> basically covered), then yes, you might do this; and then I would say
> that using "type(None)" is the correct spelling of it.

This is actual code I have:

@singledispatch
def as_color(color):
"""Convert object to QColor."""
return QtGui.QColor(color)

as_color.register(type(None), lambda x: QtGui.QColor(0, 0, 0, 0))

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Checking whether type is None

2018-07-26 Thread Stephan Houben
Op 2018-07-25, Ian Kelly schreef :

> Is there a reason for using singledispatch here rather than a simpler and
> more readable "if color is None" check?

Yes, the other 20 cases I didn't show.
And extensibility.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Using Python on a fork-less POSIX-like OS

2018-07-31 Thread Stephan Houben
Op 2018-07-29, Terry Reedy schreef :

> multiprocessing module uses 'spawn' rather than 'fork' on Windows and it 
> has an option to use 'spawn' even on *nix.  I presume the latter refers 
> to posix_spawn.  You might want to check the multiprocessing code to see 
> what *it* is doing 'under the covers'.

Actually, the only place posix_spawn is currently used in Python is in
some experimental and currently-disabled code in the posix module.

`spawn' in multiprocessing only refers to a fork/exec combination 
(on Posix, at least), as opposed to `fork' which only does a fork.

Note that posix_spawn cannot provide full subprocess functionality
(obviously preexec_fn, but also not start_new_session).
So it's a hard sell to add a posix_spawn special case there, since 
a generic fork/exec would need to be maintained anyway.

(The same reason is also why most shells such as bash, zsh, don't bother
with posix_spawn.)

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: unpacking elements in python - any tips u want to share ?

2017-08-06 Thread Stephan Houben
Hi Ganesh,

Op 2017-07-27, Ganesh Pal schreef :

> I have a list with say 7 elements  say if I need to unpack  first 3
> elements  in the list and pass it an argument to the new fuction, here is
> my elementary code

One way to do exactly what you request here is:

new_function(*my_list[:3])

Explanation:
1. my_list[:3] produces a list of the first 3 elements of my_list
2. new_function(*some_sequence) 
   (Note the * in the calling syntax!)
   is equivalent to calling new_function with all the elements of
   some_sequence individually:

   new_function(some_sequence[0], some_sequence[1], ...)

But I observe that this is slightly different from the code you
actually present:

> ...var1,var2,var3,var4,var5,var6,var7 = my_list
> ...var8 =  get_eighth_element(var1,int(var2),int(var3))

This code actually converts var2 and var3 into an integer first.
(By calling int).

Short intermezzo: I notice that you are apparently using Python 2,
because of this code:

> ….print my_list

This code would be print(my_list) in Python 3. 
If you are just learning Python and there is no particular reason you
need Python 2, I would recommend Python 3 at this point. Python 2 is
still supported but only until 2020 so you may want to invest in
something more future-proof.

Anyway, the reason I bring it up is because in Python 3 you can do:

  var1,var2,var3,*_ = my_list
  var8 =  get_eighth_element(var1,int(var2),int(var3))

This will work for any my_list which has at least 3 elements.
The syntax *_ binds the rest of the list to the variable _.
_ is a conventional name for a variable in which value you are not
interested.

If backward-compatibility with Python 2 is important, you can do instead:

  var1,var2,var3 = my_list[:3]
  var8 =  get_eighth_element(var1,int(var2),int(var3))


Finally, I would want to point out that if these fields of the list
have some specific meaning (e.g. my_list[0] is first name, my_list[1]
is last name, my_list[2] is address, etc..), then a more idiomatic
Python solution would be to define a class with named members for these
things:

  class Person:
 def __init__(self, first_name, last_name, address, city):
 self.first_name = first_name
 self.last_name = last_name
 self.address = address
 self.city = city


Greetings,

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Cross-language comparison: function map and similar

2017-08-20 Thread Stephan Houben
Op 2017-08-16, Steve D'Aprano schreef :
> Are there language implementations which evaluate the result of map()
> (or its equivalent) in some order other than the obvious left-to-right
> first-to-last sequential order? Is that order guaranteed by the
> language, or is it an implementation detail?
>
> Standard library functions implementing an explicitly "parallel map"
> or "threaded map" are also relevant. (Less interested in third-party
> libraries, unless they're practically a standard for the language in
> question.)

C++17 has such a facility in its standard library.

std::transform (and many other functions operating on sequences, but
you asked for a map() equivalent) takes an optional
"execution_policy" parameter which indicates if the operation 
should be run sequentially (the default) or can be parallellized.

See: http://en.cppreference.com/w/cpp/algorithm/transform

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-22 Thread Stephan Houben
Op 2017-08-11, Paul Rubin schreef :
> I don't think we need this since we have itertools.takewhile:
>
>   from operator import gt
>   from functools import partial
>   from itertools import takewhile
>
>   [x + 1 for x in takewhile(partial(gt,5), (0,1,2,999,3,4))]
>

No need for partial and gt.

  [x + 1 for x in takewhile((5).__gt__, (0,1,2,999,3,4))]

Basically, Haskell's infix opererator sections can often be
translated into Python by attribute access to the bound method.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-22 Thread Stephan Houben
Op 2017-08-22, Ian Kelly schreef :
> Careful! Python's dunder methods are reserved for use by Python.
> They're exposed so that we can override them. Calling them directly is
> generally considered bad style. And in this case specifically, it's
> not equivalent. 

Mmm, you are correct. That's kind of a deception, really.

I have often done things like:

  generate_id = itertools.count().__next__

but I suppose that isn't OK either, then.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-23 Thread Stephan Houben
Op 2017-08-23, Ben Finney schreef :
> Could you be convinced to instead do::
>
> import functools
> import itertools
>
> generate_id = functools.partial(next, itertools.count())

I certainly could be, but I was so far unaware of the desirability to do so.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Case-insensitive string equality

2017-09-03 Thread Stephan Houben
Op 2017-09-02, Pavol Lisy schreef :
> But problem is that if somebody like to have stable API it has to be
> changed to "do what the Unicode consortium said (at X.Y. )" :/

It is even more exciting. Presumably a reason to have case-insentivity
is to be compatible with existing popular case-insentive systems.

So here is, for your entertainment, how some of these systems work.

* Windows NTFS case-insensitive file system

  A NTFS file system contains a hidden table $UpCase which maps
  characters to their upcase variant. Note:

  1. This maps characters in the BMP *only*, so NTFS treats
 characters outside the BMP as case-insensitive.
  2. Every character can in turn only be mapped into a single
 BMP character, so ß -> SS is not possible.
  3. The table is in practice dependent on the language of the
 Windows system which created it (so a Turkish NTFS partition
 would contain i -> İ), but in theory can contain any allowed
 mapping: I can create an NTFS filesystem which maps a -> b.
  4. Older Windows versions generated tables which
 were broken for certain languages (NT 3.51/Georgian). 
 You may still have some NTFS partition with such a table lying
 around.

* macOS case-insensitive file system

  1. HFS+ is based on Unicode 3.2; this is fixed in stone because of
 backward compatibility.
  2. APFS is based on Unicode 9.0 and does normalization as well

Generally speaking, the more you learn about case normalization,
the more attractive case sensitivity looks ;-)
Also slim hope to have a single caseInsensitiveCompare function
which "Does The Right Thing"™.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: tkinter keypress events are a mess for numpad keys

2017-09-03 Thread Stephan Houben
Op 2017-08-29, Irmen de Jong schreef :

> I'll have a look at https://www.tcl.tk/man/tcl8.6/TkCmd/keysyms.htm
> but I don't have high hopes because I already tried empirically to
> figure out the distinguishing attributes of the keypress event object,
> on various platforms (Mac OS, Linux, Windows)...

I would suggest trying to reproduce the issue in a small Tcl/Tk script.
If the issue can be reproduced in Tcl I would suggest filing a bug
report at https://core.tcl.tk/tk/reportlist .

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list