pybabel: default locale is None ???
Python 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64
bit (AMD64)] on win32
>>> from datetime import timedelta
>>> from babel.dates import format_timedelta
>>> td = timedelta(seconds=39.28355172422679)
>>> format_timedelta(td)
Traceback (most recent call last):
File "", line 1, in
File "C:\Python35\lib\site-packages\babel\dates.py", line 779, in
format_timedelta
plural_form = locale.plural_form(value)
AttributeError: 'NoneType' object has no attribute 'plural_form'
>>> from babel.core import default_locale
>>> repr(default_locale('LC_CTIME'))
'None'
Why? The default_locale documentation tells this:
"Returns the system default locale for a given category, based on
environment variables."
--
https://mail.python.org/mailman/listinfo/python-list
Re: palindrome
Seymore4Head wrote: > http://www.practicepython.org/exercise/2014/03/12/06-string-lists.html > > Here is my answers. What would make it better? 1. Break the code into functions: one to generate a random string (the desired length could be a parameter) and one to check if the string is a palindrome. With that the loop will become tries = 0 while True: tries += 1 candidate = random_string(length=4) print(candidate) if is_palindrome(candidate): break print(tries, "tries") 2. If you plan to reuse these functions put the above code in a function (let's call it main), too, that you invoke with if __name__ == "__main__": main() to avoid that the code is executed when you import the module instead of running it as a script. 3. For better readability add spaces around operators. There is a tool called pep8 that will point out where you are breaking the standard Python coding conventions. 4. Minor rewrites: 4.1 Can you rewrite the while loop as a for loop? for tries in ...: ... Hint 1: you can put a while loop into a generator Hint 2: there's a ready-made solution in itertools. 4.2 Can you build the random string using a generator expression and "".join(...)? > import random > str1="" > letcount=4 > count=0 > abc='abcdefghijklmnopqrstuvwxyz' > while True: > for i in range(letcount): > a=random.choice(abc) > str1+=a > print str1 > count+=1 > if str1==str1[::-1]: > break > else: > str1="" > print "Tries= ",count > print str1 -- https://mail.python.org/mailman/listinfo/python-list
Re: palindrome
Abhiram R wrote: > Haha. Nice. Although with your length of string and the range you're > picking from,the chances of you getting a palindrome are (1/24!) :D Are you sure? >>> candidates = list(itertools.product(string.ascii_lowercase, repeat=4)) >>> len(candidates)/len([c for c in candidates if c == c[::-1]]) 676.0 That looks like one in 26**(length//2) -- https://mail.python.org/mailman/listinfo/python-list
Re: Problems using struct pack/unpack in files, and reading them.
Steven D'Aprano wrote:
>On Mon, 16 Nov 2015 05:15 pm, Gregory Ewing wrote:
>
>> Ints are not the only thing that // can be applied to:
>>
>> >>> 1.0//0.01
>> 99.0
>
>Good catch!
Hmmm. I see that the float for 0.01 _is_ slightly larger than 0.01
>>> Decimal(0.01)
Decimal('0.0120816681711721685132943093776702880859375')
But it seems that 1.0 // 0.01 is not directly equivalent to:
>>> int(1.0 / 0.01)
100
And I see that:
>>> Decimal(1.0 / 0.01)
Decimal('100')
>>> floor(1.0 / 0.01)
100
>>> 0.01 * 100.0 - 1.0
0.0
So I guess that the // operator is _very_ strict in its "floor division"
of floats, but that the "/" operator returns the nearest float value.
--
https://mail.python.org/mailman/listinfo/python-list
JOSE modules
HI! It seems there are already three modules for implementing JOSE (see RFC 7515..7520). :-/ Anyone here who has practical experience with any of them (with Python 2.7.x and preferrably with elliptic curves)? Ciao, Michael. pyjwkest https://pypi.python.org/pypi/pyjwkest JWCrypto https://pypi.python.org/pypi/jwcrypto http://jwcrypto.readthedocs.org/en/stable/index.html python-jose https://pypi.python.org/pypi/python-jose -- https://mail.python.org/mailman/listinfo/python-list
HTTPSConnection from http.client?
Hello, does anybody know how to create a HTTPS connections in python2 or python3? I tried second day to do that with http.client[1], but every time get error: from http.client import HTTPSConnection ImportError: cannot import name HTTPSConnection Where is HTTPSConnection located? Which module? I use openSUSE 13.1 x86_64. Thank you, Alex [1] https://docs.python.org/3/library/http.client.html -- https://mail.python.org/mailman/listinfo/python-list
Re: HTTPSConnection from http.client?
Alex Naumov wrote: > Hello, > > does anybody know how to create a HTTPS connections in python2 or python3? > I tried second day to do that with http.client[1], but every time get > error: > > from http.client import HTTPSConnection > ImportError: cannot import name HTTPSConnection > > > Where is HTTPSConnection located? Which module? > > I use openSUSE 13.1 x86_64. > > Thank you, > Alex > > > [1] https://docs.python.org/3/library/http.client.html Did you compile Python from source? You may be missing the development packages. If you are using the Python provided by the distribution, perhaps support for SSL is in a separate package? Note that I'm guessing, I don't have openSUSE. -- https://mail.python.org/mailman/listinfo/python-list
Re: HTTPSConnection from http.client?
Peter Otten <__pete...@web.de>: > Alex Naumov wrote: >> I tried second day to do that with http.client[1], but every time get >> error: >> >> from http.client import HTTPSConnection >> ImportError: cannot import name HTTPSConnection >> [...] >> I use openSUSE 13.1 x86_64. > > Did you compile Python from source? You may be missing the development > packages. > > If you are using the Python provided by the distribution, perhaps support > for SSL is in a separate package? > > Note that I'm guessing, I don't have openSUSE. On a minimal SLES-12-x86_64 installation: $ python3 Python 3.4.1 (default, Jun 19 2014, 14:40:48) [GCC] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from http.client import HTTPSConnection >>> Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: HTTPSConnection from http.client?
On Tue, Nov 17, 2015 at 12:22 PM, Peter Otten <__pete...@web.de> wrote: > Alex Naumov wrote: > >> Hello, >> >> does anybody know how to create a HTTPS connections in python2 or python3? >> I tried second day to do that with http.client[1], but every time get >> error: >> >> from http.client import HTTPSConnection >> ImportError: cannot import name HTTPSConnection >> >> >> Where is HTTPSConnection located? Which module? >> >> I use openSUSE 13.1 x86_64. >> >> Thank you, >> Alex >> >> >> [1] https://docs.python.org/3/library/http.client.html > > Did you compile Python from source? No, I install it as a package. > You may be missing the development > packages. Yes, it looks like some packages are not there. > On a minimal SLES-12-x86_64 installation: Can you tell me what's the name of package with http.client module please? Thank you, Alex -- https://mail.python.org/mailman/listinfo/python-list
Trouble installing Python 3.5.0
Hello, I installed 3.5, "successfully" as the installer indicated, yet the program wouldn't run, with the following error message: "...failed to start because api-ms-win-crt-runtime-I1-1-0.dll was not found. Re-installing the application may fix this problem." I reinstalled, but no luck. I noticed during installation that all the files were 64-bit (as my computer is), except one: the Launcher appears as 32-bit. Could that be the source of the problem? (The error occurs during launch, after all.) The installer I used is "python-3.5.0-amd64.exe" downloaded from Python.org directly. Could you have accidentally shipped the 32-bit Launcher with the 64-bit exe? There was one other wrinkle. When I first installed Python 3.5, I accidentally installed the 32-bit version, because it is what came up when I pressed the big yellow button for 3.5 on your site (next to the other yellow button for Python 2.7.10). In a moment of forgetfulness I didn't check for the 64-bit version. Could this have messed things up? I uninstalled a couple times, so it should have cleared things up. But maybe not? I found the 64-bit version later, but it was a bit hard to find. Why are you putting the 32-bit version out front and center, and not offering an option for the 64-bit right next to it? Best wishes, Robert Z. P.S. I'm running Vista, so my PC is a bit old, but it is in fact 64-bit, and Python 3.2 still runs fine on it. -- https://mail.python.org/mailman/listinfo/python-list
Re: HTTPSConnection from http.client?
Alex Naumov wrote: > On Tue, Nov 17, 2015 at 12:22 PM, Peter Otten <__pete...@web.de> wrote: >> Alex Naumov wrote: >> >>> Hello, >>> >>> does anybody know how to create a HTTPS connections in python2 or >>> python3? I tried second day to do that with http.client[1], but every >>> time get error: >>> >>> from http.client import HTTPSConnection >>> ImportError: cannot import name HTTPSConnection >>> >>> >>> Where is HTTPSConnection located? Which module? >>> >>> I use openSUSE 13.1 x86_64. >>> >>> Thank you, >>> Alex >>> >>> >>> [1] https://docs.python.org/3/library/http.client.html >> >> Did you compile Python from source? > > No, I install it as a package. > >> You may be missing the development >> packages. > > Yes, it looks like some packages are not there. > > >> On a minimal SLES-12-x86_64 installation: > > Can you tell me what's the name of package with http.client module please? You already have that module as it is part of the Python standard installation. You can double check by importing something that is always available. E. g. the following should succeed on your system >>> from http.client import HTTPConnection >>> What you are likely missing is ssl support. Try >>> import ssl to verify that it fails and post the traceback. Unfortunately my conclusion from Marco's post is that >>> from http.client import HTTPSConnection should succeed on your system without the need for additional packages. Something seems to be broken, but I have no idea what, sorry. -- https://mail.python.org/mailman/listinfo/python-list
cPickle.load vs. file.read+cPickle.loads on large binary files
Hello List,
I am working with relatively humongous binary files (created via cPickle),
and I stumbled across some unexpected (for me) performance differences between
two approaches I use to load those files:
1. Simply use cPickle.load(fid)
2. Read the file as binary using file.read() and then use cPickle.loads on the
resulting output
In the snippet below, the MakePickle function is a dummy function that
generates a relatively big binary file with cPickle (WARNING: around 3 GB) in
the current directory. I am using NumPy arrays to make the file big but my
original data structure is much more complicated, and things like HDF5 or
databases are currently not an option - I'd like to stay with pickles.
The ReadPickle function simply uses cPickle.load(fid) on the opened binary
file, and on my PC it takes about 2.3 seconds (approach 1).
The ReadPlusLoads function reads the file using file.read() and then use
cPickle.loads on the resulting output (approach 2). On my PC, the file.read()
process takes 15 seconds (!!!) and the cPickle.loads only 1.5 seconds.
What baffles me is the time it takes to read the file using file.read(): is
there any way to slurp it all in one go (somehow) into a string ready for
cPickle.loads without that much of an overhead?
Note that all of this has been done on Windows 7 64bit with Python 2.7 64bit,
with 16 cores and 100 GB RAM (so memory should not be a problem).
Thank you in advance for all suggestions :-) .
Andrea.
# Begin code
import os, sys
import time
import cPickle
import numpy
class Dummy(object):
def __init__(self, name):
self.name = name
self.data = numpy.random.rand(200, 600, 10)
def MakePickle():
num_objects = 300
list_of_objects = []
for index in xrange(num_objects):
dummy = Dummy('dummy_%d'%index)
list_of_objects.append(dummy)
fid = open('dummy.pkl', 'wb')
start = time.time()
out = cPickle.dumps(list_of_objects, cPickle.HIGHEST_PROTOCOL)
end = time.time()
print 'cPickle.dumps time:', end-start
start = end
fid.write(out)
end = time.time()
print 'file.write time:', end-start
fid.close()
def ReadPickle():
fid = open('dummy.pkl', 'rb')
start = time.time()
out = cPickle.load(fid)
end = time.time()
print 'cPickle.load time:', end-start
fid.close()
def ReadPlusLoads():
start = time.time()
fid = open('dummy.pkl', 'rb')
strs = fid.read()
fid.close()
end = time.time()
print 'file.read time:', end-start
start = end
out = cPickle.loads(strs)
end = time.time()
print 'cPickle.loads time:', end-start
if __name__ == '__main__':
ReadPickle()
ReadPlusLoads()
# End code
--
https://mail.python.org/mailman/listinfo/python-list
Re: cPickle.load vs. file.read+cPickle.loads on large binary files
andrea.gav...@gmail.com wrote: > Hello List, > > I am working with relatively humongous binary files (created via > cPickle), and I stumbled across some unexpected (for me) performance > differences between two approaches I use to load those files: > > 1. Simply use cPickle.load(fid) > > 2. Read the file as binary using file.read() and then use cPickle.loads on > the resulting output > > In the snippet below, the MakePickle function is a dummy function that > generates a relatively big binary file with cPickle (WARNING: around 3 GB) > in the current directory. I am using NumPy arrays to make the file big but > my original data structure is much more complicated, and things like HDF5 > or databases are currently not an option - I'd like to stay with pickles. > > The ReadPickle function simply uses cPickle.load(fid) on the opened binary > file, and on my PC it takes about 2.3 seconds (approach 1). > > The ReadPlusLoads function reads the file using file.read() and then use > cPickle.loads on the resulting output (approach 2). On my PC, the > file.read() process takes 15 seconds (!!!) and the cPickle.loads only 1.5 > seconds. > > What baffles me is the time it takes to read the file using file.read(): > is there any way to slurp it all in one go (somehow) into a string ready > for cPickle.loads without that much of an overhead? > > Note that all of this has been done on Windows 7 64bit with Python 2.7 > 64bit, with 16 cores and 100 GB RAM (so memory should not be a problem). > > Thank you in advance for all suggestions :-) . > > Andrea. > > if __name__ == '__main__': > ReadPickle() > ReadPlusLoads() Do you get roughly the same times when you execute ReadPlusLoads() before ReadPIckle()? -- https://mail.python.org/mailman/listinfo/python-list
Re: cPickle.load vs. file.read+cPickle.loads on large binary files
Hi Peter, On Tuesday, November 17, 2015 at 3:14:57 PM UTC+1, Peter Otten wrote: > Andrea Gavana wrote: > > > Hello List, > > > > I am working with relatively humongous binary files (created via > > cPickle), and I stumbled across some unexpected (for me) performance > > differences between two approaches I use to load those files: > > > > 1. Simply use cPickle.load(fid) > > > > 2. Read the file as binary using file.read() and then use cPickle.loads on > > the resulting output > > > > In the snippet below, the MakePickle function is a dummy function that > > generates a relatively big binary file with cPickle (WARNING: around 3 GB) > > in the current directory. I am using NumPy arrays to make the file big but > > my original data structure is much more complicated, and things like HDF5 > > or databases are currently not an option - I'd like to stay with pickles. > > > > The ReadPickle function simply uses cPickle.load(fid) on the opened binary > > file, and on my PC it takes about 2.3 seconds (approach 1). > > > > The ReadPlusLoads function reads the file using file.read() and then use > > cPickle.loads on the resulting output (approach 2). On my PC, the > > file.read() process takes 15 seconds (!!!) and the cPickle.loads only 1.5 > > seconds. > > > > What baffles me is the time it takes to read the file using file.read(): > > is there any way to slurp it all in one go (somehow) into a string ready > > for cPickle.loads without that much of an overhead? > > > > Note that all of this has been done on Windows 7 64bit with Python 2.7 > > 64bit, with 16 cores and 100 GB RAM (so memory should not be a problem). > > > > Thank you in advance for all suggestions :-) . > > > > Andrea. > > > > if __name__ == '__main__': > > ReadPickle() > > ReadPlusLoads() > > Do you get roughly the same times when you execute ReadPlusLoads() before > ReadPIckle()? Thank you for your answer. I do get similar timings when I swap the two functions, and specifically still 15 seconds to read the file via file.read() and 2.4 seconds (more or less as before) via cPickle.load(fid). I thought that the order of operations might be an issue but apparently that was not the case... Andrea. -- https://mail.python.org/mailman/listinfo/python-list
Which type should be used when testing static structure appartenance
Hello,
I saw the following retweet by Raymond Hettinger in this morning:
https://twitter.com/sanityinc/status/666485814214287360
Programming tip: many of those arrays and hashes in your code
should actually be sets. Match data structures to data
constraints!
I saw just in time because in a review I wrote something like this:
if operator not in ('where', 'not where')
and my colleague proposed that I should use a list instead of a tuple.
But reading the mentioned tweet I tend to think that a set would be a
better choice.
What are your opinion on this issue (I realize it's not something of
the utmost importance but rather a "philosophical" question).
--
Nicolas Évrard - B2CK SPRL
E-mail/Jabber: nicolas.evr...@b2ck.com
Tel: +32 472 54 46 59
Website: http://www.b2ck.com/
--
https://mail.python.org/mailman/listinfo/python-list
Re: Trouble installing Python 3.5.0
> Hello, > > I installed 3.5, "successfully" as the installer indicated, yet the program > wouldn't run, with the following error message: "...failed to start because > api-ms-win-crt-runtime-I1-1-0.dll was not found. Re-installing the > application may fix this problem." I reinstalled, but no luck. I think Vista and XP was supported up to Python 3.4. The newest Python 3.5 does not support Vista or XP. -- https://mail.python.org/mailman/listinfo/python-list
Re: Trouble installing Python 3.5.0
>> Hello, >> >> I installed 3.5, "successfully" as the installer indicated, yet the program >> wouldn't run, with the following error message: "...failed to start because >> api-ms-win-crt-runtime-I1-1-0.dll was not found. Re-installing the >> application may fix this problem." I reinstalled, but no luck. > I think Vista and XP was supported up to Python 3.4. The newest Python > 3.5 does not support Vista or XP. https://docs.python.org/3.5/whatsnew/3.5.html You will find this under "Unsupported Operating Systems" section. Although Vista is not mentioned there, but Python 3.5 is built with Microsoft Visual Studio 2015, and its runtime library is AFAIK does not work under Vista. -- https://mail.python.org/mailman/listinfo/python-list
Re: cPickle.load vs. file.read+cPickle.loads on large binary files
On Wed, Nov 18, 2015 at 1:20 AM, wrote: > Thank you for your answer. I do get similar timings when I swap the two > functions, and specifically still 15 seconds to read the file via file.read() > and 2.4 seconds (more or less as before) via cPickle.load(fid). > > I thought that the order of operations might be an issue but apparently that > was not the case... What if you call one of them twice and then the other? Just trying to rule out any possibility that it's a caching problem. On my Linux box, running 2.7.9 64-bit, the two operations take roughly the same amount of time (1.8 seconds for load vs 1s to read and 0.8 to loads). Are you able to run this off a RAM disk or something? Most curious. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Which type should be used when testing static structure appartenance
On Wed, Nov 18, 2015 at 1:27 AM, Nicolas Évrard wrote:
> I saw just in time because in a review I wrote something like this:
>
>if operator not in ('where', 'not where')
>
> and my colleague proposed that I should use a list instead of a tuple.
> But reading the mentioned tweet I tend to think that a set would be a
> better choice.
>
> What are your opinion on this issue (I realize it's not something of
> the utmost importance but rather a "philosophical" question).
Definitely a set. I don't know why it would be better to use a list;
there's no advantage here for the list. "x [not] in some_set"
accurately represents the concept of "any of these will match". With a
two-element list/tuple/set, I doubt there's going to be any
significant performance difference, but if anything, I would expect
the tuple to be the fastest (because it can be stored as a literal),
and the other two need to be built; but CPython's optimizer has me
beat there - they're all stored as literals (the list becomes a tuple,
the set becomes a frozenset), meaning there's basically no difference.
So you're free to use the one that expresses the concept of "is this
part of this set of strings", which is the set.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: cPickle.load vs. file.read+cPickle.loads on large binary files
Hi Chris, On Tuesday, November 17, 2015 at 4:20:34 PM UTC+1, Chris Angelico wrote: > On Wed, Nov 18, 2015 at 1:20 AM, Andrea Gavana wrote: > > Thank you for your answer. I do get similar timings when I swap the two > > functions, and specifically still 15 seconds to read the file via > > file.read() and 2.4 seconds (more or less as before) via cPickle.load(fid). > > > > I thought that the order of operations might be an issue but apparently > > that was not the case... > > What if you call one of them twice and then the other? Just trying to > rule out any possibility that it's a caching problem. > > On my Linux box, running 2.7.9 64-bit, the two operations take roughly > the same amount of time (1.8 seconds for load vs 1s to read and 0.8 to > loads). Are you able to run this off a RAM disk or something? > > Most curious. Thank you for taking the time to run my little script. I have now run it with multiple combinations of calls (twice the first then the other, then viceversa, then alternate between the two functions multiple times, then three times the second and once the first, ...) with no luck at all. The file.read() line of code takes always at minimum 14 seconds (in all the trials I have done), while the cPickle.load call ranges between 2.3 and 2.5 seconds. I am puzzled with no end... Might there be something funny with my C libraries that use fread? I'm just shooting in the dark. I have a standard Python installation on Windows, nothing fancy :-( Andrea. -- https://mail.python.org/mailman/listinfo/python-list
Re: HTTPSConnection from http.client?
Hello Peter, thanks for your reply. >>> import ssl Works well in python2 and 3. Maybe somebody know another way to create a SSL connection (username/password)? I just need to log in and log out. Thanks, Alex On Tue, Nov 17, 2015 at 2:24 PM, Peter Otten <__pete...@web.de> wrote: > Alex Naumov wrote: > >> On Tue, Nov 17, 2015 at 12:22 PM, Peter Otten <__pete...@web.de> wrote: >>> Alex Naumov wrote: >>> Hello, does anybody know how to create a HTTPS connections in python2 or python3? I tried second day to do that with http.client[1], but every time get error: from http.client import HTTPSConnection ImportError: cannot import name HTTPSConnection Where is HTTPSConnection located? Which module? I use openSUSE 13.1 x86_64. Thank you, Alex [1] https://docs.python.org/3/library/http.client.html >>> >>> Did you compile Python from source? >> >> No, I install it as a package. >> >>> You may be missing the development >>> packages. >> >> Yes, it looks like some packages are not there. >> >> >>> On a minimal SLES-12-x86_64 installation: >> >> Can you tell me what's the name of package with http.client module please? > > You already have that module as it is part of the Python standard > installation. You can double check by importing something that is always > available. E. g. the following should succeed on your system > from http.client import HTTPConnection > > What you are likely missing is ssl support. Try > import ssl > > to verify that it fails and post the traceback. > > Unfortunately my conclusion from Marco's post is that > from http.client import HTTPSConnection > > should succeed on your system without the need for additional packages. > Something seems to be broken, but I have no idea what, sorry. > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: cPickle.load vs. file.read+cPickle.loads on large binary files
andrea.gav...@gmail.com wrote: > Hi Chris, > > On Tuesday, November 17, 2015 at 4:20:34 PM UTC+1, Chris Angelico wrote: >> On Wed, Nov 18, 2015 at 1:20 AM, Andrea Gavana wrote: >> > Thank you for your answer. I do get similar timings when I swap the two >> > functions, and specifically still 15 seconds to read the file via >> > file.read() and 2.4 seconds (more or less as before) via >> > cPickle.load(fid). >> > >> > I thought that the order of operations might be an issue but apparently >> > that was not the case... >> >> What if you call one of them twice and then the other? Just trying to >> rule out any possibility that it's a caching problem. >> >> On my Linux box, running 2.7.9 64-bit, the two operations take roughly >> the same amount of time (1.8 seconds for load vs 1s to read and 0.8 to >> loads). Are you able to run this off a RAM disk or something? >> >> Most curious. > > > Thank you for taking the time to run my little script. I have now run it > with multiple combinations of calls (twice the first then the other, then > viceversa, then alternate between the two functions multiple times, then > three times the second and once the first, ...) with no luck at all. > > The file.read() line of code takes always at minimum 14 seconds (in all > the trials I have done), while the cPickle.load call ranges between 2.3 > and 2.5 seconds. > > I am puzzled with no end... Might there be something funny with my C > libraries that use fread? I'm just shooting in the dark. I have a standard > Python installation on Windows, nothing fancy :-( Perhaps there is a size threshold? You could experiment with different block sizes in the following f.read() replacement: def read_chunked(f, size=2**20): read = functools.partial(f.read, size) return "".join(iter(read, "")) -- https://mail.python.org/mailman/listinfo/python-list
Re: Which type should be used when testing static structure appartenance
On 17 November 2015 at 14:27, Nicolas Évrard wrote:
> Hello,
>
> I saw the following retweet by Raymond Hettinger in this morning:
>
>https://twitter.com/sanityinc/status/666485814214287360
>
>Programming tip: many of those arrays and hashes in your code
>should actually be sets. Match data structures to data
>constraints!
>
> I saw just in time because in a review I wrote something like this:
>
>if operator not in ('where', 'not where')
>
> and my colleague proposed that I should use a list instead of a tuple.
> But reading the mentioned tweet I tend to think that a set would be a
> better choice.
>
> What are your opinion on this issue (I realize it's not something of
> the utmost importance but rather a "philosophical" question).
Conceptually it should be a set. Really it makes little difference though.
I think that what the tip is really referring to is a situation where
you have a larger data structure. For example asking if x in y has
different algorithmic performance when y is a large tuple/list than a
set. This is because a set is a hash-table with O(1) average lookup
cost and a tuple/list needs to be scanned linearly having O(N) average
cost. This can be very costly and can often be seen in beginner code
where someone does something like:
for x in stuff:
if x in alist:
# do stuff
Here the "in" operator loops over alist once for each item in stuff
(until it finds a match). If there are M things in stuff and N things
in alist then this is O(M*N) items to compare. If we convert alist to
a set at the start of the loop then we have:
aset = set(alist)
for xin stuff:
if x in aset:
# do stuff
Here the set constructor scans alist which is O(N) and then for each
of M things in stuff we do an O(1) lookup in aset. So the result is
O(M+N) meaning linear rather than quadratic performance. This can be a
significant difference especially for something so trivially easy to
do.
--
Oscar
--
https://mail.python.org/mailman/listinfo/python-list
Re: Trouble installing Python 3.5.0
On Tue, Nov 17, 2015 at 8:51 AM, Nagy László Zsolt wrote: > I think Vista and XP was supported up to Python 3.4. The newest Python > 3.5 does not support Vista or XP. Vista is still supported in Python 3.5, but it requires the Universal CRT update, which requires the latest Service Pack for Vista (see here: https://support.microsoft.com/en-us/kb/2999226). If your Vista is up to date, you should have no problem. It could be argued that if you're not up to date you're not supported by Microsoft and thus not supported by Python, but that may be stretching things a bit. -- Zach -- https://mail.python.org/mailman/listinfo/python-list
Help on savefig parameters
Hi,
I find the parameters of savefig function has the similar format of that of
main(*argc, **argv) in C. I have tried with savefig("temp.pdf", format='pdf'),
and it works. I get the help content of savefig() as below.
But I cannot understand why they also give:
savefig(fname, dpi=None, facecolor='w', edgecolor='w', ...
For me, it looks like the first item, i.e. 'args' is missing.
Could you explain it to me?
Thanks,
savefig(*args, **kwargs)
Save the current figure.
Call signature::
savefig(fname, dpi=None, facecolor='w', edgecolor='w',
orientation='portrait', papertype=None, format=None,
transparent=False, bbox_inches=None, pad_inches=0.1,
frameon=None)
--
https://mail.python.org/mailman/listinfo/python-list
Re: cPickle.load vs. file.read+cPickle.loads on large binary files
Hi Peter, On Tuesday, November 17, 2015 at 4:57:57 PM UTC+1, Peter Otten wrote: > Andrea Gavana wrote: > > > Hi Chris, > > > > On Tuesday, November 17, 2015 at 4:20:34 PM UTC+1, Chris Angelico wrote: > >> On Wed, Nov 18, 2015 at 1:20 AM, Andrea Gavana wrote: > >> > Thank you for your answer. I do get similar timings when I swap the two > >> > functions, and specifically still 15 seconds to read the file via > >> > file.read() and 2.4 seconds (more or less as before) via > >> > cPickle.load(fid). > >> > > >> > I thought that the order of operations might be an issue but apparently > >> > that was not the case... > >> > >> What if you call one of them twice and then the other? Just trying to > >> rule out any possibility that it's a caching problem. > >> > >> On my Linux box, running 2.7.9 64-bit, the two operations take roughly > >> the same amount of time (1.8 seconds for load vs 1s to read and 0.8 to > >> loads). Are you able to run this off a RAM disk or something? > >> > >> Most curious. > > > > > > Thank you for taking the time to run my little script. I have now run it > > with multiple combinations of calls (twice the first then the other, then > > viceversa, then alternate between the two functions multiple times, then > > three times the second and once the first, ...) with no luck at all. > > > > The file.read() line of code takes always at minimum 14 seconds (in all > > the trials I have done), while the cPickle.load call ranges between 2.3 > > and 2.5 seconds. > > > > I am puzzled with no end... Might there be something funny with my C > > libraries that use fread? I'm just shooting in the dark. I have a standard > > Python installation on Windows, nothing fancy :-( > > Perhaps there is a size threshold? You could experiment with different block > sizes in the following f.read() replacement: > > def read_chunked(f, size=2**20): > read = functools.partial(f.read, size) > return "".join(iter(read, "")) Thank you for the suggestion. I have used the read_chunked function in my experiments now and I can report a nice improvements - I have tried various chunk sizes, from 2**10 to 2**31-1, and in general the optimum lies around size=2**22, although it is essentially flat from 2**20 up to 2**30 - with some interesting spikes at 45 seconds for 2**14 and 2**15 (see table below). Using your suggestion, I got it down to 3.4 seconds (on average). Still at least twice slower than cPickle.load, but better. What I find most puzzling is that a pure file.read() (or your read_chunked variation) should normally be much faster than a cPickle.load (which does so many more things than just reading a file), shouldn't it? Timing table: Size (power of 2) Read Time (seconds) 10 9.14 11 8.59 12 7.67 13 5.70 14 46.06 15 45.00 16 24.80 17 14.23 18 8.95 19 5.58 20 3.41 21 3.39 22 3.34 23 3.39 24 3.39 25 3.42 26 3.43 27 3.44 28 3.48 29 3.59 30 3.72 -- https://mail.python.org/mailman/listinfo/python-list
Re: Help on savefig parameters
On Tue, 17 Nov 2015 08:31:08 -0800, fl wrote:
> Hi,
> I find the parameters of savefig function has the similar format of that
> of main(*argc, **argv) in C. I have tried with savefig("temp.pdf",
> format='pdf'),
> and it works. I get the help content of savefig() as below.
> But I cannot understand why they also give:
>
> savefig(fname, dpi=None, facecolor='w', edgecolor='w', ...
>
> For me, it looks like the first item, i.e. 'args' is missing.
> Could you explain it to me?
>
> Thanks,
>
>
> savefig(*args, **kwargs)
> Save the current figure.
>
> Call signature::
>
> savefig(fname, dpi=None, facecolor='w', edgecolor='w',
> orientation='portrait', papertype=None, format=None,
> transparent=False, bbox_inches=None, pad_inches=0.1,
> frameon=None)
Numpy/scipy/matplotlib are in large part autogenerated code from the
underlying libraries; that's where the default *args, **kwargs arguments
are coming from. The signature given in the documentation is the one
that actually matters.
Prefer the web documentation to the inline docs for that entire set of
libraries; it'll make your life easier.
--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order. See above to fix.
--
https://mail.python.org/mailman/listinfo/python-list
Re: cPickle.load vs. file.read+cPickle.loads on large binary files
andrea.gav...@gmail.com wrote: >> > I am puzzled with no end... Might there be something funny with my C >> > libraries that use fread? I'm just shooting in the dark. I have a >> > standard Python installation on Windows, nothing fancy :-( >> >> Perhaps there is a size threshold? You could experiment with different >> block sizes in the following f.read() replacement: >> >> def read_chunked(f, size=2**20): >> read = functools.partial(f.read, size) >> return "".join(iter(read, "")) > > > Thank you for the suggestion. I have used the read_chunked function in my > experiments now and I can report a nice improvements - I have tried > various chunk sizes, from 2**10 to 2**31-1, and in general the optimum > lies around size=2**22, although it is essentially flat from 2**20 up to > 2**30 - with some interesting spikes at 45 seconds for 2**14 and 2**15 > (see table below). > > Using your suggestion, I got it down to 3.4 seconds (on average). Still at > least twice slower than cPickle.load, but better. > > What I find most puzzling is that a pure file.read() (or your read_chunked > variation) should normally be much faster than a cPickle.load (which does > so many more things than just reading a file), shouldn't it? That would have been my expectation, too. I had a quick look into the fileobject.c source and didn't see anything that struck me as suspicious. I think you should file a bug report so that an expert can check if there is an underlying problem in Python or if it is a matter of the OS. > Timing table: > > Size (power of 2) Read Time (seconds) > 109.14 > 118.59 > 127.67 > 135.70 > 1446.06 > 1545.00 > 1624.80 > 1714.23 > 188.95 > 195.58 > 203.41 > 213.39 > 223.34 > 233.39 > 243.39 > 253.42 > 263.43 > 273.44 > 283.48 > 293.59 > 303.72 -- https://mail.python.org/mailman/listinfo/python-list
PEP 484 stubs with generic types
Playing around a bit with PEP 484, I annotated a function that returns an asyncio.Future: import asyncio def get_future() -> asyncio.Future[int]: future = asyncio.Future() future.set_result(42) return future The problem with this is that in Python 3.5, asyncio.Future can't be used as a generic type. Fortunately, the typeshed repository provides a stub that defines it as a generic type: https://github.com/python/typeshed/blob/master/stdlib/3.4/asyncio/futures.pyi This is fine for running a static type checker, but it still creates a problem when actually running the code. The real asyncio.Future doesn't support indexing, and so the annotation causes a TypeError at runtime. I could write the annotation as a forward reference, avoiding the TypeError: def get_future() -> "asyncio.Future[int]": ... But PEP 484 stipulates that a forward reference "should evaluate without errors once the module has been fully loaded", so this is invalid. Is there any advice for this case? How are generic types meant to be used with stub files? -- https://mail.python.org/mailman/listinfo/python-list
Is there any reason to introduce this intermediate variable (sz)?
Hi, I find the following code snippet, which is useful in my project: n_iter = 50 sz = (n_iter,) # size of array x = -0.37727 z = np.random.normal(x,0.1,size=sz) Q = 1e-5 # process variance # allocate space for arrays xhat=np.zeros(sz) P=np.zeros(sz) I learn Python now and the above code seems from an experienced author. The curious thing to me is the variable 'sz'. I have check np.zeros(shape, ..) shape : int or sequence of ints The introduced 'sz' is a tuple. If n_iter function is similar to a constant in C, I don't see the reason for 'sz'. In fact, 'n_iter' is an int, which fits the below np.zeros(shape) correctly. Could you see something useful with variable 'sz'? Thanks, -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there any reason to introduce this intermediate variable (sz)?
In fl writes: > correctly. Could you see something useful with variable 'sz'? 'sz' is fewer characters than '(n_iter,)', which may make your code easier to read. The np.zeros() function explicitly accepts an 'int or sequence of ints', so you don't specifically need a sequence. Is the same true for the 'size' keyword argument of np.random.normal()? -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there any reason to introduce this intermediate variable (sz)?
On Tuesday, November 17, 2015 at 4:03:05 PM UTC-5, John Gordon wrote: > In fl <@gmail.com> > writes: > > > correctly. Could you see something useful with variable 'sz'? > > 'sz' is fewer characters than '(n_iter,)', which may make your code easier > to read. > > The np.zeros() function explicitly accepts an 'int or sequence of ints', > so you don't specifically need a sequence. Is the same true for the > 'size' keyword argument of np.random.normal()? > > -- > John Gordon A is for Amy, who fell down the stairs > @panix.com B is for Basil, assaulted by bears > -- Edward Gorey, "The Gashlycrumb Tinies" Hi, I get the following for the third parameter of np.random.normal(): size : int or tuple of ints, optional I still don't see the necessity of 'sz'. Thanks, -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there any reason to introduce this intermediate variable (sz)?
In fl writes: > I still don't see the necessity of 'sz'. Thanks, sz isn't required. You can use (n_iter,) in place of sz. However, as I posted earlier, sz is shorter so it might make your code easier to read. Using sz can also lead to easier code maintenance. If the contents of the tuple were ever to change, it would be much easier to simply change it in once place (the definition of sz), rather than having to edit several different occurrences of (n_iter,) in your code. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there any reason to introduce this intermediate variable (sz)?
On 17/11/2015 21:27, fl wrote: On Tuesday, November 17, 2015 at 4:03:05 PM UTC-5, John Gordon wrote: In fl <@gmail.com> writes: correctly. Could you see something useful with variable 'sz'? 'sz' is fewer characters than '(n_iter,)', which may make your code easier to read. The np.zeros() function explicitly accepts an 'int or sequence of ints', so you don't specifically need a sequence. Is the same true for the 'size' keyword argument of np.random.normal()? -- John Gordon A is for Amy, who fell down the stairs @panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" Hi, I get the following for the third parameter of np.random.normal(): size : int or tuple of ints, optional I still don't see the necessity of 'sz'. Thanks, I don't see the necessity for you to bombard this list with question after question without bothering to try and provide your own answers first. Did your last skivvy die of overwork? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Mapping between python packages and distro packages?
Hi, Do you know if there is a library to match a python package (from PyPI) and find the right debian/redhat packages ? Thank you, Stephane -- Stéphane Wirtel - http://wirtel.be - @matrixise -- https://mail.python.org/mailman/listinfo/python-list
Re: Mapping between python packages and distro packages?
Stephane Wirtel writes: > Do you know if there is a library to match a python package (from PyPI) > and find the right debian/redhat packages ? What would count as “the right package”? Do you mean “the package that has that PyPI distribution URL in its ‘debian/watch’ configuration”? Do you mean “the package that names that PyPI distribution in its ‘debian/copyright’ “Source” field”? Or something else? The answers will differ depending on exactly what question is being asked of the data. If you can define exactly what query you want to perform, perhaps a solution can be offered. -- \ “The Vatican is not a state.… a state must have territory. This | `\ is a palace with gardens, about as big as an average golf | _o__) course.” —Geoffrey Robertson, 2010-09-18 | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Mapping between python packages and distro packages?
On Wed, Nov 18, 2015 at 3:33 PM, Ben Finney wrote: > What would count as “the right package”? > > Do you mean “the package that has that PyPI distribution URL in its > ‘debian/watch’ configuration”? > > Do you mean “the package that names that PyPI distribution in its > ‘debian/copyright’ “Source” field”? > > Or something else? I don't know how the program would detect it, but I'd be thinking "the one where 'sudo apt-get install PACKAGENAME' gets the same code that 'pip install THING' gets". In a lot of cases, PACKAGENAME will simply be python-THING or python3-THING, eg python3-sqlalchemy, python3-scipy, python3-bs4; as a human, that's what I'd try first. But having a program recognize this would be hugely beneficial. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Writing SOME class methods in C
Hello, I'm trying to implement some (but not all) methods of a Python class in C. What I've found on the Net is: - how to implement entire modules in C so that I can import that module and use the C functions (successfully done it, too). - how to implement entire classes in C But I can't find any examples of modules which consist of a mixture of C and Python, nor modules that define classes of which some members are implemented in C, others in Python. Of course, once I have the "mixture" bit figured out I could just define wrapper class methods that call C functions (functions impleneted in C, that is). But I would find it rather elegant if the C function could access the class members directly. The fact that the C extension functions have a mandatory "PyObject *self" parameter tells me that this must be somehow possible, but I don't know how. I'm sure that many of the plethora of Python extension modules out there must use the technique that I'm looking for, but I don't even know where to start looking. Pointers are welcome. Thanks, robert -- https://mail.python.org/mailman/listinfo/python-list
