Re: Python is not bad ;-)

[Steven D'Aprano]

On Thu, 30 Apr 2015 09:30 pm, Cecil Westerhof wrote:

> Tail recursion would nice to have also.

People coming from functional languages like Lisp and Haskell often say
that, but how many recursive algorithms naturally take a tail-call form?
Not that many.

I suppose that it would be nice if Python let you optionally use tail-call
optimization, but that might be tricky in practice.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looks like Python 3 files do not confirm to Python 3

[Steven D'Aprano]

On Fri, 1 May 2015 04:27 pm, Cecil Westerhof wrote:

> On my system in:
> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

ndg is a third-party package, not part of the Python 3 standard library.

> it says:
> try:
> from ndg.httpsclient.subj_alt_name import SubjectAltName
> from pyasn1.codec.der import decoder as der_decoder
> SUBJ_ALT_NAME_SUPPORT = True
> except ImportError, e:
[...]
> which gives:
> File
> 
> "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
> line 17
>   except ImportError, e:

I think you're missing the last line of the error. I'm guessing it was
probably NameError: name 'e' is not defined.


> Does my system have outdated files, or are there still Python 3 files
> that do not conform to Python 3?

I would say, either you have accidentally installed a Python 2 file in your
Python 3 library, or it is simply a bug in ndg.


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is my implementation of happy number OK

[Jon Ribbens]

On 2015-04-30, Dave Angel  wrote:
> On 04/30/2015 07:31 PM, Jon Ribbens wrote:
>> On 2015-04-30, Dave Angel  wrote:
>>> But the real reason I didn't like it was it produced a much larger
>>> set of happy_numbers, which could clog memory a lot sooner.  For
>>> 10**7 items, I had 3250 happy members, and 19630 unhappy.  And Jon
>>> had 1418854 happy members.
>>
>> Er, what? You're complaining that mine is less efficient by not
>> producing the wrong output?
>
> It's not intended as a criticism;  you solved a different problem.  The 
> problem Cecil was solving was to determine if a particular number is 
> happy.  The problem you solved was to make a list of all values under a 
> particular limit that are happy.
>
> Both produce identical results for the Cecil purpose, and yours is 
> faster if one wants all the values.  But if one wants a sampling of 
> values, his function will fetch them quickly, and even if you want them 
> all, his function will use much less memory.

I must admit, I'm still not understanding. If you want to know only
whether or not int(1e7) is happy then the calculation takes no
measurable time or memory.
-- 
https://mail.python.org/mailman/listinfo/python-list

Rewriting to Python 3

[Cecil Westerhof]

On my system I have:
PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
in:
/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

In Python 3 that gives:
TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_values'

How should I rewrite this?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looks like Python 3 files do not confirm to Python 3

[Cecil Westerhof]

Op Friday 1 May 2015 08:27 CEST schreef Cecil Westerhof:

> On my system in:
> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> it says:
> try:
> from ndg.httpsclient.subj_alt_name import SubjectAltName
> from pyasn1.codec.der import decoder as der_decoder
> SUBJ_ALT_NAME_SUPPORT = True
> except ImportError, e:
> SUBJ_ALT_NAME_SUPPORT = False
> SUBJ_ALT_NAME_SUPPORT_MSG = (
> 'SubjectAltName support is disabled - check pyasn1 package '
> 'installation to enable'
> )
> import warnings
> warnings.warn(SUBJ_ALT_NAME_SUPPORT_MSG)
>
> which gives: File
> "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
> line 17 except ImportError, e:

It is solved by making it:
except (ImportError) as e:

The same for:
/usr/lib/python3.4/site-packages/ndg/httpsclient/subj_alt_name.py

I would not mind to solve those and others if I could contribute them
to the Python community. What is the procedure for that?

>
> Does my system have outdated files, or are there still Python 3
> files that do not conform to Python 3?
>
> The only real python program I use at the moment uses:
> import urllib3.contrib.pyopenssl
>
> and then you get this error.
>
>
> So another reason to stay with Python 2. (While still writing code
> that works in Python3.)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Rewriting to Python 3

[Ben Finney]

Cecil Westerhof  writes:

> On my system I have:
> PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
> in:
> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

Are you in contact with the author of that third-party library? Why is
it in your Python 3.4 site-packages?

> In Python 3 that gives:
> TypeError: unsupported operand type(s) for +: 'dict_keys' and 
> 'dict_values'
>
> How should I rewrite this?

My attempt:

PARSER_RE_STR = "/({keys}|{values})=".format(
keys=DN_LUT.keys(), values=DN_LUT.values())

Explicit is better than implicit, especially in mini-languages like
string interpolation.

-- 
 \  “Often, the surest way to convey misinformation is to tell the |
  `\   strict truth.” —Mark Twain, _Following the Equator_ |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looks like Python 3 files do not confirm to Python 3

[Cecil Westerhof]

Op Friday 1 May 2015 09:00 CEST schreef Steven D'Aprano:

> On Fri, 1 May 2015 04:27 pm, Cecil Westerhof wrote:
>
>> On my system in:
>> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> ndg is a third-party package, not part of the Python 3 standard
> library.

Oops, barking up to the wrong tree. Sorry.


>> it says:
>> try:
>> from ndg.httpsclient.subj_alt_name import SubjectAltName
>> from pyasn1.codec.der import decoder as der_decoder
>> SUBJ_ALT_NAME_SUPPORT = True
>> except ImportError, e:
> [...]
>> which gives: File
>> "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
>> line 17 except ImportError, e:
>
> I think you're missing the last line of the error. I'm guessing it
> was probably NameError: name 'e' is not defined.

Nope:
except ImportError, e:
  ^
SyntaxError: invalid syntax

It is solved by:
except (ImportError) as e:

But then:
PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_values'


>> Does my system have outdated files, or are there still Python 3
>> files that do not conform to Python 3?
>
> I would say, either you have accidentally installed a Python 2 file
> in your Python 3 library, or it is simply a bug in ndg.

I will contact the ndg people.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Jedi is now a static analysis library

[Steven D'Aprano]

Jedi, the refactoring and static analysis tool, is now able to be used as a
library for your own code. See the announcement here:

https://mail.python.org/pipermail/code-quality/2015-April/000534.html


Jedi itself:

https://github.com/davidhalter/jedi/



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Rewriting to Python 3

[Peter Otten]

Cecil Westerhof wrote:

> On my system I have:
> PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
> in:
> /usr/lib/python3.4/site-
packages/ndg/httpsclient/ssl_peer_verification.py
> 
> In Python 3 that gives:
> TypeError: unsupported operand type(s) for +: 'dict_keys' and
> 'dict_values'
> 
> How should I rewrite this?


There's a tool called 2to3 -- it doesn't produce perfect code but it can 
cope with the mechanical changes:
 
$ cat demo.py
d = dict("aA bB cC".split())

try:
print d.keys() + d.values()
except Exception, e:
print e
$ 2to3 -w demo.py
[...]
$ cat demo.py
d = dict("aA bB cC".split())

try:
print(list(d.keys()) + list(d.values()))
except Exception as e:
print(e)
$ python3 demo.py
['c', 'a', 'b', 'C', 'A', 'B']
$

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python is not bad ;-)

[Cecil Westerhof]

Op Friday 1 May 2015 09:03 CEST schreef Steven D'Aprano:

> On Thu, 30 Apr 2015 09:30 pm, Cecil Westerhof wrote:
>
>> Tail recursion would nice to have also.
>
> People coming from functional languages like Lisp and Haskell often
> say that, but how many recursive algorithms naturally take a
> tail-call form? Not that many.

When I was playing with Scala there where a few cases where tail
recursion made a significant performance boost.

Is some time ago, so I do not remember which.


> I suppose that it would be nice if Python let you optionally use
> tail-call optimization, but that might be tricky in practice.

That is the difference between Scala and Clojure. Scala does it
silently, while by Clojure you have to say you want it. When a chance
breaks tail recursion Scala just compiles, but your code becomes
slower, Clojure does not compile. I prefer the Clojure variant.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looks like Python 3 files do not confirm to Python 3

[Mark Lawrence]

On 01/05/2015 08:16, Cecil Westerhof wrote:

Op Friday 1 May 2015 08:27 CEST schreef Cecil Westerhof:


On my system in:
/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

it says:
try:
from ndg.httpsclient.subj_alt_name import SubjectAltName
from pyasn1.codec.der import decoder as der_decoder
SUBJ_ALT_NAME_SUPPORT = True
except ImportError, e:
SUBJ_ALT_NAME_SUPPORT = False
SUBJ_ALT_NAME_SUPPORT_MSG = (
'SubjectAltName support is disabled - check pyasn1 package'
'installation to enable'
)
import warnings
warnings.warn(SUBJ_ALT_NAME_SUPPORT_MSG)

which gives: File
"/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
line 17 except ImportError, e:


It is solved by making it:
 except (ImportError) as e:

The same for:
 /usr/lib/python3.4/site-packages/ndg/httpsclient/subj_alt_name.py

I would not mind to solve those and others if I could contribute them
to the Python community. What is the procedure for that?



You'll have to find out who supports ndg as it's not standard Python. 
The big clue is the site-packages folder name, that's used for third 
party packages.  From searching I think it's this 
https://github.com/cedadev/ndg_httpsclient/ but please don't quote me on 
that.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Rewriting to Python 3

[Chris Angelico]

On Fri, May 1, 2015 at 5:22 PM, Cecil Westerhof  wrote:
> On my system I have:
> PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
> in:
> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> In Python 3 that gives:
> TypeError: unsupported operand type(s) for +: 'dict_keys' and 
> 'dict_values'
>
> How should I rewrite this?

The keys() and values() methods used to return lists, now they return
views. If you explicitly listify them, it should work; alternatively,
pipe-join each one separately and then pipe-join the result:

PARSER_RE_STR = '/(%s|%s)=' % ('|'.join(DN_LUT.keys()),
'|'.join(DN_LUT.values()))

Note that this is slightly different from the above; in the case of an
empty dict, the original produces empty parentheses, but my
alternative will place a junk pipe in there.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Rewriting to Python 3

[Mark Lawrence]

On 01/05/2015 08:22, Cecil Westerhof wrote:

On my system I have:
 PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
in:
 /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

In Python 3 that gives:
 TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_values'

How should I rewrite this?



I'd run the entire package through the 2to3 tool that's part of the 
standard library, or uninstall that package and get an up to date 
version that actually supports Python 3.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Is my implementation of happy number OK

[Steven D'Aprano]

On Fri, 1 May 2015 05:23 pm, Jon Ribbens wrote:

> On 2015-04-30, Dave Angel  wrote:
>> On 04/30/2015 07:31 PM, Jon Ribbens wrote:
>>> On 2015-04-30, Dave Angel  wrote:
 But the real reason I didn't like it was it produced a much larger
 set of happy_numbers, which could clog memory a lot sooner.  For
 10**7 items, I had 3250 happy members, and 19630 unhappy.  And Jon
 had 1418854 happy members.
>>>
>>> Er, what? You're complaining that mine is less efficient by not
>>> producing the wrong output?
>>
>> It's not intended as a criticism;  you solved a different problem.  The
>> problem Cecil was solving was to determine if a particular number is
>> happy.  The problem you solved was to make a list of all values under a
>> particular limit that are happy.
>>
>> Both produce identical results for the Cecil purpose, and yours is
>> faster if one wants all the values.  But if one wants a sampling of
>> values, his function will fetch them quickly, and even if you want them
>> all, his function will use much less memory.
> 
> I must admit, I'm still not understanding. If you want to know only
> whether or not int(1e7) is happy then the calculation takes no
> measurable time or memory.

Oh, I'm sure somebody would be able to measure it...

Rather than 10**7, how about trying (10**500 + 2). Is it happy?

Using the Python code from Wikipedia:
https://en.wikipedia.org/wiki/Happy_number

SQUARE = dict([(c, int(c)**2) for c in "0123456789"])
def is_happy(n):
  while (n > 1) and (n != 4):
n = sum(SQUARE[d] for d in str(n))
  return n == 1


I can calculate whether n=10**500 + 2 is happy in less than a millisecond on
my computer. Using your version doesn't even work, as it would require
significantly more than 1000 terrabytes of RAM just to hold the results. I
don't have that much memory.

Your version is a reasonable answer to a different question, but it doesn't
scale to very large inputs.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looks like Python 3 files do not confirm to Python 3

[Cecil Westerhof]

Op Friday 1 May 2015 10:05 CEST schreef Mark Lawrence:

> On 01/05/2015 08:16, Cecil Westerhof wrote:
>> Op Friday 1 May 2015 08:27 CEST schreef Cecil Westerhof:
>>
>>> On my system in:
>>> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>>>
>>> it says:
>>> try:
>>> from ndg.httpsclient.subj_alt_name import SubjectAltName
>>> from pyasn1.codec.der import decoder as der_decoder
>>> SUBJ_ALT_NAME_SUPPORT = True
>>> except ImportError, e:
>>> SUBJ_ALT_NAME_SUPPORT = False
>>> SUBJ_ALT_NAME_SUPPORT_MSG = (
>>> 'SubjectAltName support is disabled - check pyasn1 package'
>>> 'installation to enable'
>>> )
>>> import warnings
>>> warnings.warn(SUBJ_ALT_NAME_SUPPORT_MSG)
>>>
>>> which gives: File
>>> "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
>>> line 17 except ImportError, e:
>>
>> It is solved by making it:
>> except (ImportError) as e:
>>
>> The same for:
>> /usr/lib/python3.4/site-packages/ndg/httpsclient/subj_alt_name.py
>>
>> I would not mind to solve those and others if I could contribute
>> them to the Python community. What is the procedure for that?
>>
>
> You'll have to find out who supports ndg as it's not standard
> Python. The big clue is the site-packages folder name, that's used
> for third party packages. From searching I think it's this
> https://github.com/cedadev/ndg_httpsclient/ but please don't quote
> me on that.

Yeah, I was barking up to the wrong tree.

I found an email address in the sources. The copyright is from 2012,
so hopefully it is still supported.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Am I missing something here? ipaddress vs socket

[the . loeki]

Hi all,

Given the following code:

import ipaddress
import socket

ip = ipaddress.ip_address(mystring)
sock_family = ip.
socket = socket.socket(sock_family, socket.SOCK_STREAM)

Am I crazy or is this undoable?

sock.AF_INET == 2
sock.AF_INET6 == 10
ip.version == 4 or 6


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Am I missing something here? ipaddress vs socket

[Chris Angelico]

On Fri, May 1, 2015 at 7:42 PM,   wrote:
> Hi all,
>
> Given the following code:
>
> import ipaddress
> import socket
>
> ip = ipaddress.ip_address(mystring)
> sock_family = ip.
> socket = socket.socket(sock_family, socket.SOCK_STREAM)
>
> Am I crazy or is this undoable?
>
> sock.AF_INET == 2
> sock.AF_INET6 == 10
> ip.version == 4 or 6

Are you trying to look up a name to get an address? Or just look up an
address? The easiest way would be to use a ternary if:

sock_family = sock.AF_INET if ip.version == 4 else sock.AF_INET6

But you may find it convenient to use a dedicated function for
establishing a connection, which could look up an  or A record for
a name, then proceed through all addresses, attempting connections in
turn. I'm fairly sure one exists in Python, but I can't right now
remember the name.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

GAE environment differences

[Robin Becker]

A user reports getting KeyError/ImportError when executing GAE dev/live code 
looking like



import os
...
fn = os.path.expanduser(os.path.join('~','.reportlab_settings'))
..

This works fine in cpython; so I assume that the GAE environment must be doing 
module resolution differently.


Is that the case or is it that os.path or os.path.expanduser doesn't exist?
--
Robin Becker

--
https://mail.python.org/mailman/listinfo/python-list

Re: Am I missing something here? ipaddress vs socket

[Alain Ketterlin]

the.lo...@gmail.com writes:

> Given the following code:
>
> import ipaddress
> import socket
>
> ip = ipaddress.ip_address(mystring)
> sock_family = ip.
> socket = socket.socket(sock_family, socket.SOCK_STREAM)
>
> Am I crazy or is this undoable?
>
> sock.AF_INET == 2
> sock.AF_INET6 == 10
> ip.version == 4 or 6

Check the value of the "version" attribute. Or use socket.getaddrinfo(),
which is more convenient if all you need is an address parser.

-- Alain.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Am I missing something here? ipaddress vs socket

[Ronald van Zantvoort]

Hi Chris & Alain, thanks for  responding :)

On Friday, 1 May 2015 12:11:20 UTC+2, Chris Angelico  wrote:
> On Fri, May 1, 2015 at 7:42 PM,  Ronald van Zantvoort wrote:
> > Hi all,
> >
> > Given the following code:
> >
> > import ipaddress
> > import socket
> >
> > ip = ipaddress.ip_address(mystring)
> > sock_family = ip.
> > socket = socket.socket(sock_family, socket.SOCK_STREAM)
> >
> > Am I crazy or is this undoable?
> >
> > sock.AF_INET == 2
> > sock.AF_INET6 == 10
> > ip.version == 4 or 6
> 
> Are you trying to look up a name to get an address? Or just look up an
> address? The easiest way would be to use a ternary if:
> 
> sock_family = sock.AF_INET if ip.version == 4 else sock.AF_INET6

mystring is already an IP address, sorry for not mentioning that.
The point is that ternary if; I've been scouring around the internet and 
there's a million different variations on the same idea. 
I'm completely flabberghasted it's not a simple property within the object 
though, e.g. ip.sock_family.

Was this a deliberate decision by the Python devs or can I try an RFE for that?

> But you may find it convenient to use a dedicated function for
> establishing a connection, which could look up an  or A record for
> a name, then proceed through all addresses, attempting connections in
> turn. I'm fairly sure one exists in Python, but I can't right now
> remember the name.
> 
> ChrisA

Well you've already got socket.create_connection(), which (I think) does 
perform that lookup.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ImportError with pickle (Python 2.7.9), possibly platform dependent

[Ben Sizer]

On Thursday, 30 April 2015 01:45:05 UTC+1, Chris Angelico  wrote:
> On Thu, Apr 30, 2015 at 2:01 AM, Ben Sizer wrote:
> > 1) There clearly is a module named OMDBMap, and it's importable - it's 
> > there in the 2nd line of the traceback.
> >
> > Does anybody have any suggestions on how I can go about debugging this? Or 
> > refactoring it to avoid whatever is happening here?
> 
> Are you half way through importing it when this load() call happens?
> That might cause some issues.

No, we already imported OMDBMap at the top of OMDBSetup.

> Has your current directory been changed anywhere in there?

Good question. It turns out that the current directory seems to be $HOME when 
loading, but is the script directory during saving. This will be because the 
Linux server is running under mod_wsgi, whereas we run the save script 
in-place. Our Windows and Mac tests run via Flask's built-in server so the 
working directory is likely to be the same whether we're running the script 
that does pickle.dump or the whole app that does pickle.load.

> What happens if you catch this exception and print out sys.modules at
> that point?

Another good question, and this gives us the answer. The module lists are quite 
different, as I'd expect because the load happens in the context of the full 
application whereas the dump happens as a standalone script. But literally 
every module that is in the 'before dump' list is in the 'before load' list - 
except OMDBMap. Like the error says! What /is/ in the 'before load' list 
however is "my_wsgi_app.OMDBMap". The module has been imported, but the pickle 
algorithm is unable to reconcile the module in the WSGI app's namespace with 
the module referenced in the pickle file.

So... I don't know how to fix this, but I do now know why it fails, and I have 
a satisfactory answer for why it is acting differently on the Linux server (and 
that is just because that is the only one running under WSGI). Two out of three 
isn't bad!

Thanks,
-- 
Ben Sizer

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: l = range(int(1E9))

[Thomas 'PointedEars' Lahn]

Cecil Westerhof wrote:

> By the way: I also see python3.4 and python3.4m. Any idea where the m
> stands for?

I googled for “python3.4m” and found as second result



In a nutshell: python3.4m was built with configure option “--with-pymalloc” 
which causes the resulting implementation to use “Pymalloc, a specialized 
object allocator written by Vladimir Marangozov, […] a feature added to 
Python 2.1. Pymalloc is intended to be faster than the system malloc() and 
to have less memory overhead for allocation patterns typical of Python 
programs. The allocator uses C's malloc() function to get large pools of 
memory and then fulfills smaller memory requests from these pools.”

See also:  (first result)

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Am I missing something here? ipaddress vs socket

[Chris Angelico]

On Fri, May 1, 2015 at 8:41 PM, Ronald van Zantvoort
 wrote:
> mystring is already an IP address, sorry for not mentioning that.
> The point is that ternary if; I've been scouring around the internet and 
> there's a million different variations on the same idea.
> I'm completely flabberghasted it's not a simple property within the object 
> though, e.g. ip.sock_family.
>
> Was this a deliberate decision by the Python devs or can I try an RFE for 
> that?

You could ask. At very worst, you'll find someone explain to you why
it shouldn't be done. I suggest using the python-ideas mailing list;
people there are usually reasonably friendly, though occasionally you
find a caustic response to a dumb idea like "Python should use braces
for indentation" :)

>> But you may find it convenient to use a dedicated function for
>> establishing a connection, which could look up an  or A record for
>> a name, then proceed through all addresses, attempting connections in
>> turn. I'm fairly sure one exists in Python, but I can't right now
>> remember the name.
>>
> Well you've already got socket.create_connection(), which (I think) does 
> perform that lookup.

Yeah, I went looking for it and came across that, but wasn't sure if
that was it.

Now I look more closely (ie: look at the actual source code), I see
that it probably is the function I was thinking of. Might need a docs
patch to make it clear that this iterates over getaddrinfo() and
attempts them all in succession.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ImportError with pickle (Python 2.7.9), possibly platform dependent

[Chris Angelico]

On Fri, May 1, 2015 at 9:01 PM, Ben Sizer  wrote:
> Another good question, and this gives us the answer. The module lists are 
> quite different, as I'd expect because the load happens in the context of the 
> full application whereas the dump happens as a standalone script. But 
> literally every module that is in the 'before dump' list is in the 'before 
> load' list - except OMDBMap. Like the error says! What /is/ in the 'before 
> load' list however is "my_wsgi_app.OMDBMap". The module has been imported, 
> but the pickle algorithm is unable to reconcile the module in the WSGI app's 
> namespace with the module referenced in the pickle file.
>
> So... I don't know how to fix this, but I do now know why it fails, and I 
> have a satisfactory answer for why it is acting differently on the Linux 
> server (and that is just because that is the only one running under WSGI). 
> Two out of three isn't bad!
>

Cool! That's part way. So, can you simply stuff OMDBMap into
sys.modules prior to loading? It might be a bit of a hack, but it
should work for testing, at least. Conversely, you could change the
dump script to import via the name my_wsgi_app to make it consistent.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: GAE environment differences

[Chris Angelico]

On Fri, May 1, 2015 at 8:29 PM, Robin Becker  wrote:
> A user reports getting KeyError/ImportError when executing GAE dev/live code
> looking like
>
>
> import os
> ...
> fn = os.path.expanduser(os.path.join('~','.reportlab_settings'))
> ..
>
> This works fine in cpython; so I assume that the GAE environment must be
> doing module resolution differently.
>
> Is that the case or is it that os.path or os.path.expanduser doesn't exist?

Best thing to do is to ask the user to post the complete traceback.
You might need to use "import os.path" but normally I would expect
that not to be an issue.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Fast way of extracting files from various folders

[subhabrata . banerji]

Dear Group,

I have several millions of documents in several folders and subfolders in my 
machine.
I tried to write a script as follows, to extract all the .doc files and to 
convert them in text, but it seems it is taking too much of time. 

import os
from fnmatch import fnmatch
import win32com.client
import zipfile, re
def listallfiles2(n):
root = 'C:\Cand_Res'
pattern = "*.doc"
list1=[]
for path, subdirs, files in os.walk(root):
for name in files:
if fnmatch(name, pattern):
file_name1=os.path.join(path, name)
if ".doc" in file_name1:
#EXTRACTING ONLY .DOC FILES
if ".docx" not in file_name1:
#print "It is A Doc file$$:",file_name1
try:
doc = win32com.client.GetObject(file_name1)
text = doc.Range().Text
text1=text.encode('ascii','ignore')
text_word=text1.split()
#print "Text for Document File Is:",text1
list1.append(text_word)
print "It is a Doc file"
except:
print "DOC ISSUE"

But it seems it is taking too much of time, to convert to text and to append to 
list. Is there any way I may do it fast? I am using Python2.7 on Windows 7 
Professional Edition. Apology for any indentation error. 

If any one may kindly suggest a solution.

Regards,
Subhabrata Banerjee. 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ImportError with pickle (Python 2.7.9), possibly platform dependent

[Peter Otten]

Ben Sizer wrote:

> On Thursday, 30 April 2015 01:45:05 UTC+1, Chris Angelico  wrote:
>> On Thu, Apr 30, 2015 at 2:01 AM, Ben Sizer wrote:
>> > 1) There clearly is a module named OMDBMap, and it's importable - it's
>> > there in the 2nd line of the traceback.
>> >
>> > Does anybody have any suggestions on how I can go about debugging this?
>> > Or refactoring it to avoid whatever is happening here?
>> 
>> Are you half way through importing it when this load() call happens?
>> That might cause some issues.
> 
> No, we already imported OMDBMap at the top of OMDBSetup.
> 
>> Has your current directory been changed anywhere in there?
> 
> Good question. It turns out that the current directory seems to be $HOME
> when loading, but is the script directory during saving. This will be
> because the Linux server is running under mod_wsgi, whereas we run the
> save script in-place. Our Windows and Mac tests run via Flask's built-in
> server so the working directory is likely to be the same whether we're
> running the script that does pickle.dump or the whole app that does
> pickle.load.
> 
>> What happens if you catch this exception and print out sys.modules at
>> that point?
> 
> Another good question, and this gives us the answer. The module lists are
> quite different, as I'd expect because the load happens in the context of
> the full application whereas the dump happens as a standalone script. But
> literally every module that is in the 'before dump' list is in the 'before
> load' list - except OMDBMap. Like the error says! What /is/ in the 'before
> load' list however is "my_wsgi_app.OMDBMap". The module has been imported,
> but the pickle algorithm is unable to reconcile the module in the WSGI
> app's namespace with the module referenced in the pickle file.
> 
> So... I don't know how to fix this, but I do now know why it fails, and I
> have a satisfactory answer for why it is acting differently on the Linux
> server (and that is just because that is the only one running under WSGI).
> Two out of three isn't bad!

How about moving OMDBMap.py into the parent folder of my_wsgi_app.__file__ 
or any other folder in sys.path?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: l = range(int(1E9))

[Cecil Westerhof]

Op Friday 1 May 2015 13:15 CEST schreef Thomas Lahn:

> Cecil Westerhof wrote:
>
>> By the way: I also see python3.4 and python3.4m. Any idea where the
>> m stands for?
>
> I googled for “python3.4m” and found as second result

Eh, I could/should have done that myself. :-(
Nice that you do not burn me. :-D


> 
>
> In a nutshell: python3.4m was built with configure option
> “--with-pymalloc” which causes the resulting implementation to use
> “Pymalloc, a specialized object allocator written by Vladimir
> Marangozov, […] a feature added to Python 2.1. Pymalloc is intended
> to be faster than the system malloc() and to have less memory
> overhead for allocation patterns typical of Python programs. The
> allocator uses C's malloc() function to get large pools of memory
> and then fulfills smaller memory requests from these pools.”
>
> See also:  (first
> result)

Something to look in later. Looks interesting, but have a ‘few’ things
that are more important.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

[no subject]

[newton avetisyan]

Sent from Windows Mail-- 
https://mail.python.org/mailman/listinfo/python-list

Re: GAE environment differences

[Robin Becker]

On 01/05/2015 13:15, Chris Angelico wrote:

On Fri, May 1, 2015 at 8:29 PM, Robin Becker  wrote:


Best thing to do is to ask the user to post the complete traceback.
You might need to use "import os.path" but normally I would expect
that not to be an issue.





jamesbynd said:

here's a traceback:

```
#!python

ImportError: No module named pwd
(12 additional frame(s) were not displayed)
...
  File 
"/base/data/home/apps/e~yt-maggi-2015-eu/testing.383971015313322618/vendor/xhtml2pdf/context.py",
 line 5, in 
from reportlab.lib.styles import ParagraphStyle
  File 
"/base/data/home/apps/e~yt-maggi-2015-eu/testing.383971015313322618/vendor/reportlab/lib/styles.py",
 line 28, in 
from reportlab.rl_config import canvas_basefontname as _baseFontName, 
baseUnderlineProportion as _baseUnderlineProportion
  File 
"/base/data/home/apps/e~yt-maggi-2015-eu/testing.383971015313322618/vendor/reportlab/rl_config.py",
 line 131, in 
_startUp()
  File 
"/base/data/home/apps/e~yt-maggi-2015-eu/testing.383971015313322618/vendor/reportlab/rl_config.py",
 line 99, in _startUp
d = os.path.expanduser(d)   #appengine fails with KeyError
  File "python2.7/posixpath.py", line 268, in expanduser
import pwd
```
the user suggests that even though claims are made that you can use a 
filesystem, but stuff like pwd is missing. Apparently the user module has no 
meaning, but there is a users module? I guess I'll need to keep patching 
reportlab when GAE users find these glitches.

--
Robin Becker

--
https://mail.python.org/mailman/listinfo/python-list

Re: GAE environment differences

[Chris Angelico]

On Fri, May 1, 2015 at 11:04 PM, Robin Becker  wrote:
>>   File "python2.7/posixpath.py", line 268, in expanduser
>> import pwd
>> ```
>
> the user suggests that even though claims are made that you can use a
> filesystem, but stuff like pwd is missing. Apparently the user module has no
> meaning, but there is a users module? I guess I'll need to keep patching
> reportlab when GAE users find these glitches.

According to my Python installation, the pwd module is built-in. But
you could bypass its usage by making sure $HOME is set before calling
expanduser.

What happens if you fire up Python and just type "import pwd"? Does
that work? If not, does it give any more useful error message?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: l = range(int(1E9))

[Michael Torrie]

On 04/30/2015 10:19 PM, Cecil Westerhof wrote:
>> I must also confess to being highly impressed, it's a breath of
>> fresh air having an apprentice Pythonista who is looking at doing
>> things the Pythonic way :)
> 
> When in Rome, do as the Romans do.
> 
> Besides: there probably is a reason for the Pythonic way.

You maybe surprised, then, by how many people that stop by this list
wanting to program Java, C# or C++ in Python and get frustrated and
upset when the list tries to steer them in a Pythonic direction.  Most
of them leave upset and never return to the list or Python.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: GAE environment differences

[Peter Otten]

Chris Angelico wrote:

> On Fri, May 1, 2015 at 11:04 PM, Robin Becker  wrote:
>>>   File "python2.7/posixpath.py", line 268, in expanduser
>>> import pwd
>>> ```
>>
>> the user suggests that even though claims are made that you can use a
>> filesystem, but stuff like pwd is missing. Apparently the user module has
>> no meaning, but there is a users module? I guess I'll need to keep
>> patching reportlab when GAE users find these glitches.
> 
> According to my Python installation, the pwd module is built-in. But
> you could bypass its usage by making sure $HOME is set before calling
> expanduser.
> 
> What happens if you fire up Python and just type "import pwd"? Does
> that work? If not, does it give any more useful error message?

I found (guess how)

https://cloud.google.com/appengine/kb/libraries

which states:

"""
As a result, these are the C modules that are not usable with the Python 2.7 
runtime:

ctypes
sqlite
ssl/_ssl
fcntl
spwd
pwd
grp
syslog
select
_socket
"""

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Am I missing something here? ipaddress vs socket

[Ronald van Zantvoort]

On Friday, 1 May 2015 14:06:34 UTC+2, Chris Angelico  wrote:
> On Fri, May 1, 2015 at 8:41 PM, Ronald van Zantvoort wrote:
> > mystring is already an IP address, sorry for not mentioning that.
> > The point is that ternary if; I've been scouring around the internet and 
> > there's a million different variations on the same idea.
> > I'm completely flabberghasted it's not a simple property within the object 
> > though, e.g. ip.sock_family.
> >
> > Was this a deliberate decision by the Python devs or can I try an RFE for 
> > that?
> 
> You could ask. At very worst, you'll find someone explain to you why
> it shouldn't be done. I suggest using the python-ideas mailing list;
> people there are usually reasonably friendly, though occasionally you
> find a caustic response to a dumb idea like "Python should use braces
> for indentation" :)

There are no dumb ideas, just dumb people ;)

But seriously, I'll go see over there, thanks :) It's just that I'm amazed that 
this isn't in there yet; it's 2 simple properties added in for convenience and 
IMHO very Pythonic to be able to do aforementioned code.

Thanks again pplz :)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Am I missing something here? ipaddress vs socket

[Chris Angelico]

On Fri, May 1, 2015 at 11:37 PM, Ronald van Zantvoort
 wrote:
> But seriously, I'll go see over there, thanks :) It's just that I'm amazed 
> that this isn't in there yet; it's 2 simple properties added in for 
> convenience and IMHO very Pythonic to be able to do aforementioned code.
>

There are lots of things that are so obvious they get added with
little argument, but someone still has to think of them :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Am I missing something here? ipaddress vs socket

[Ronald van Zantvoort]

On Friday, 1 May 2015 15:46:32 UTC+2, Chris Angelico  wrote:
> On Fri, May 1, 2015 at 11:37 PM, Ronald van Zantvoort wrote:
> > But seriously, I'll go see over there, thanks :) It's just that I'm amazed 
> > that this isn't in there yet; it's 2 simple properties added in for 
> > convenience and IMHO very Pythonic to be able to do aforementioned code.
> >
> 
> There are lots of things that are so obvious they get added with
> little argument, but someone still has to think of them :)
> 
> ChrisA

Hehehe right you are. I've posted an idea to python-ideas, we'll see how it 
goes from there.
Thanks again!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: l = range(int(1E9))

[Cecil Westerhof]

Op Friday 1 May 2015 15:25 CEST schreef Michael Torrie:

> On 04/30/2015 10:19 PM, Cecil Westerhof wrote:
>>> I must also confess to being highly impressed, it's a breath of
>>> fresh air having an apprentice Pythonista who is looking at doing
>>> things the Pythonic way :)
>>
>> When in Rome, do as the Romans do.
>>
>> Besides: there probably is a reason for the Pythonic way.
>
> You maybe surprised, then, by how many people that stop by this list
> wanting to program Java, C# or C++ in Python and get frustrated and
> upset when the list tries to steer them in a Pythonic direction.
> Most of them leave upset and never return to the list or Python.

I am not really surprised. I know that this is how it often works. :-(
But I like to be different. :-D

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is my implementation of happy number OK

[Ian Kelly]

On Fri, May 1, 2015 at 2:27 AM, Steven D'Aprano
 wrote:
> Rather than 10**7, how about trying (10**500 + 2). Is it happy?
>
> Using the Python code from Wikipedia:
> https://en.wikipedia.org/wiki/Happy_number
>
> SQUARE = dict([(c, int(c)**2) for c in "0123456789"])
> def is_happy(n):
>   while (n > 1) and (n != 4):
> n = sum(SQUARE[d] for d in str(n))
>   return n == 1
>
>
> I can calculate whether n=10**500 + 2 is happy in less than a millisecond on
> my computer.

Not really the most exciting example, since the following number in
the sequence would be 5. But a random sequence of 500 non-zero digits
doesn't seem to take substantially longer.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Rewriting to Python 3

[Cecil Westerhof]

Op Friday 1 May 2015 09:22 CEST schreef Cecil Westerhof:

> On my system I have: PARSER_RE_STR = '/(%s)=' %
> '|'.join(DN_LUT.keys() + DN_LUT.values()) in:
> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> In Python 3 that gives: TypeError: unsupported operand type(s) for
> +: 'dict_keys' and 'dict_values'
>
> How should I rewrite this?

I am not the only person that has a problem:
https://github.com/cedadev/ndg_httpsclient/issues/1

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Rewriting to Python 3

[Cecil Westerhof]

Op Friday 1 May 2015 16:41 CEST schreef Cecil Westerhof:

> Op Friday 1 May 2015 09:22 CEST schreef Cecil Westerhof:
>
>> On my system I have: PARSER_RE_STR = '/(%s)=' %
>> '|'.join(DN_LUT.keys() + DN_LUT.values()) in:
>> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>>
>> In Python 3 that gives: TypeError: unsupported operand type(s) for
>> +: 'dict_keys' and 'dict_values'
>>
>> How should I rewrite this?
>
> I am not the only person that has a problem:
> https://github.com/cedadev/ndg_httpsclient/issues/1

Just personally contacting does make a difference. He made a patch. I
am going to test a later today.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fast way of extracting files from various folders

[Irmen de Jong]

On 1-5-2015 14:28, subhabrata.bane...@gmail.com wrote:
> Dear Group,
> 
> I have several millions of documents in several folders and subfolders in my 
> machine.
> I tried to write a script as follows, to extract all the .doc files and to 
> convert them in text, but it seems it is taking too much of time. 
> 

[snip]

> But it seems it is taking too much of time, to convert to text and to append 
> to list. Is there any way I may do it fast? I am using Python2.7 on Windows 7 
> Professional Edition. Apology for any indentation error. 
> 
> If any one may kindly suggest a solution.

Have you profiled and identified the part of your script that is slow?

On first sight though your python code, while not optimal, contains no immediate
performance issues. It is likely the COM interop call to Winword and getting 
the text
via that interface that is slow. Imagine opening word for "several million 
documents",
no wonder it doesn't perform.

Investigate tools like antiword, wv, docx2txt. I suspect they're quite a bit 
faster than
relying on Word itself.


Irmen


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python is not bad ;-)

[Christian Gollwitzer]

Am 01.05.15 um 09:03 schrieb Steven D'Aprano:

On Thu, 30 Apr 2015 09:30 pm, Cecil Westerhof wrote:


Tail recursion would nice to have also.


People coming from functional languages like Lisp and Haskell often say
that, but how many recursive algorithms naturally take a tail-call form?
Not that many.


That is because tailcall optimization is used in functional languages 
mainly as a replacement for loops. IOW, in non-functional languages you 
can simply use an infinite loop to do the same (in most cases).



I suppose that it would be nice if Python let you optionally use tail-call
optimization, but that might be tricky in practice.



In Tcl 8.6 there is an explicit "tailcall something" command which calls 
something without creating a new stack frame, i.e. it returns to the 
caller after something returns. Useful sometimes, but more a 
micro-optimization than a vital feature.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is my implementation of happy number OK

[Peter Otten]

Ian Kelly wrote:

> On Fri, May 1, 2015 at 2:27 AM, Steven D'Aprano
>  wrote:
>> Rather than 10**7, how about trying (10**500 + 2). Is it happy?
>>
>> Using the Python code from Wikipedia:
>> https://en.wikipedia.org/wiki/Happy_number
>>
>> SQUARE = dict([(c, int(c)**2) for c in "0123456789"])
>> def is_happy(n):
>>   while (n > 1) and (n != 4):
>> n = sum(SQUARE[d] for d in str(n))
>>   return n == 1
>>
>>
>> I can calculate whether n=10**500 + 2 is happy in less than a millisecond
>> on my computer.
> 
> Not really the most exciting example, since the following number in
> the sequence would be 5. But a random sequence of 500 non-zero digits
> doesn't seem to take substantially longer.

I may be missing something, but isn't the '+ 2' in '10**500 + 2' just a 
distraction? 10**500 would bring across nicely what the approach Steven took 
from Wikipeda can do in one iteration where Jon's cannot even find enough 
memory.

Also, with 500 digits and 0 <= d <= 9 

sum(d*d for d in digits) <= 81 * 500

should hold, i. e. on the second iteration the maximum number of digits is 
already down to five in the worst case. By repeating this simple reasoning 
you can tell that all interesting examples must be below 1000 as 
num_digits(3*81) == 3. 

A computer that cannot calculate a lookup table with all 3-digit cases 
should be almost as hard to find as the one needed by Jon ;)


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: l = range(int(1E9))

[Terry Reedy]

On 5/1/2015 1:04 AM, Cecil Westerhof wrote:


By the way: I also see python3.4 and python3.4m. Any idea where the m
stands for?


I never heard of that in 18 years of Python, and thought it must be an 
error, but putting 'python3.4b' into google search return this.


https://stackoverflow.com/questions/16675865/difference-between-python3-and-python3m-executibles

See first answer.
--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list