Re: Interesting performance question

2019-09-29 Thread Terry Reedy
On 9/29/2019 9:45 AM, Eryk Sun wrote: On 9/29/19, Anthony Flury via Python-list wrote: Using python 3.6 building a tuple like this : my_tuple = tuple([x*x for x in range(1,1000)]) The list comprehension is implemented internally as a function that builds and returns the list. This function

Re: Interesting performance question

2019-09-29 Thread Eryk Sun
On 9/29/19, Anthony Flury via Python-list wrote: > > Using python 3.6 building a tuple like this : > > my_tuple = tuple([x*x for x in range(1,1000)]) The list comprehension is implemented internally as a function that builds and returns the list. This function creates an empty list and loops over

Interesting performance question

2019-09-29 Thread Anthony Flury via Python-list
I have just noticed an oddity : Using python 3.6 building a tuple like this : my_tuple = tuple([x*x for x in range(1,1000)]) is about 1/3 quicker than     my_tuple = tuple(x*x for x in range(1,1000)) Measurements : $  python3 -m timeit 'my_tuple = tuple([x*x for x in range(1,1000)])' 1

Re: Multiprocessing performance question

2019-02-21 Thread DL Neil
George: apologies for mis-identifying yourself as OP. Israel: On 22/02/19 6:04 AM, Israel Brewster wrote: Actually not a ’toy example’ at all. It is simply the first step in gridding some data I am working with - a problem that is solved by tools like SatPy, but unfortunately I can’t use SatPy

Re: Multiprocessing performance question

2019-02-21 Thread Israel Brewster
Actually not a ’toy example’ at all. It is simply the first step in gridding some data I am working with - a problem that is solved by tools like SatPy, but unfortunately I can’t use SatPy because it doesn’t recognize my file format, and you can’t load data directly. Writing a custom file import

Re: Multiprocessing performance question

2019-02-20 Thread george trojan
I don't know whether this is a toy example, having grid of this size is not uncommon. True, it would make more sense to do distribute more work on each box, if there was any. One has to find a proper balance, as with many other things in life. I simply responded to a question by the OP. George O

Re: Multiprocessing performance question

2019-02-20 Thread DL Neil
George On 21/02/19 1:15 PM, george trojan wrote: def create_box(x_y): return geometry.box(x_y[0] - 1, x_y[1], x_y[0], x_y[1] - 1) x_range = range(1, 1001) y_range = range(1, 801) x_y_range = list(itertools.product(x_range, y_range)) grid = list(map(create_box, x_y_range)) Which creates

Re: Multiprocessing performance question

2019-02-20 Thread george trojan
def create_box(x_y): return geometry.box(x_y[0] - 1, x_y[1], x_y[0], x_y[1] - 1) x_range = range(1, 1001) y_range = range(1, 801) x_y_range = list(itertools.product(x_range, y_range)) grid = list(map(create_box, x_y_range)) Which creates and populates an 800x1000 “grid” (represented as a fl

Re: Multiprocessing performance question

2019-02-18 Thread Israel Brewster
> On Feb 18, 2019, at 6:37 PM, Ben Finney wrote: > > I don't have anything to add regarding your experiments with > multiprocessing, but: > > Israel Brewster writes: > >> Which creates and populates an 800x1000 “grid” (represented as a flat >> list at this point) of “boxes”, where a box is a

Re: Multiprocessing performance question

2019-02-18 Thread Ben Finney
I don't have anything to add regarding your experiments with multiprocessing, but: Israel Brewster writes: > Which creates and populates an 800x1000 “grid” (represented as a flat > list at this point) of “boxes”, where a box is a > shapely.geometry.box(). This takes about 10 seconds to run. Thi

Multiprocessing performance question

2019-02-18 Thread Israel Brewster
I have the following code running in python 3.7: def create_box(x_y): return geometry.box(x_y[0] - 1, x_y[1], x_y[0], x_y[1] - 1) x_range = range(1, 1001) y_range = range(1, 801) x_y_range = list(itertools.product(x_range, y_range)) grid = list(map(create_box, x_y_range)) Which creates and

Re: performance question: dictionary or list, float or string?

2008-12-04 Thread bkamrani
On Dec 4, 1:28 pm, alex23 <[EMAIL PROTECTED]> wrote: > On Dec 4, 8:12 pm, [EMAIL PROTECTED] wrote: > > > About the piece of code you posted, there is something I don't > > understand. > > >         for i, line in data: > > > where data is a file object. Is it legal to write that? > > I believe it r

Re: performance question: dictionary or list, float or string?

2008-12-04 Thread alex23
On Dec 4, 8:12 pm, [EMAIL PROTECTED] wrote: > About the piece of code you posted, there is something I don't > understand. > >         for i, line in data: > > where data is a file object. Is it legal to write that? > I believe it results in "too many values to unpack" or do I miss > something? >F

Re: performance question: dictionary or list, float or string?

2008-12-04 Thread bkamrani
About the piece of code you posted, there is something I don't understand. for i, line in data: where data is a file object. Is it legal to write that? I believe it results in "too many values to unpack" or do I miss something? /Ben On Dec 4, 10:26 am, [EMAIL PROTECTED] wrote: > Matt, r

Re: performance question: dictionary or list, float or string?

2008-12-04 Thread bkamrani
Matt, really thanks for your comments! Even thogh it was not a direct answer to my questions, I like your coding style very much and I think you have a good point. About the number of line in the file, because I get that info from another in advance. Therefore I thought it could be hard coded. BT

Re: performance question: dictionary or list, float or string?

2008-12-04 Thread bkamrani
Thanks for your questions. Here come some answer below. On Dec 2, 2:50 pm, Steven D'Aprano <[EMAIL PROTECTED] cybersource.com.au> wrote: > On Tue, 02 Dec 2008 03:41:29 -0800,bkamraniwrote: > > Hi Python gurus! > > I'm going to read in an Ascii file containing float numbers in rows and > > columns

Re: performance question: dictionary or list, float or string?

2008-12-02 Thread Matimus
On Dec 2, 3:51 am, [EMAIL PROTECTED] wrote: > I forgot to mention that I did a simple timeit test which doesn't > show > significant runtime difference 3.5 sec for dictionary case and 3.48 > for > list case. > > def read_as_dictionary(): >     fil = open('myDataFile', 'r') >     forces = {} >     f

Re: performance question: dictionary or list, float or string?

2008-12-02 Thread Steven D'Aprano
On Tue, 02 Dec 2008 03:41:29 -0800, bkamrani wrote: > Hi Python gurus! > I'm going to read in an Ascii file containing float numbers in rows and > columns (say 10 columns 50 rows) for further numerical process. > Which format is best to save them in, eg, dictionary, list, or numpy > array when

Re: performance question: dictionary or list, float or string?

2008-12-02 Thread bkamrani
I forgot to mention that I did a simple timeit test which doesn't show significant runtime difference 3.5 sec for dictionary case and 3.48 for list case. def read_as_dictionary(): fil = open('myDataFile', 'r') forces = {} for region in range(25): forces[region] = {} for s

performance question: dictionary or list, float or string?

2008-12-02 Thread bkamrani
Hi Python gurus! I'm going to read in an Ascii file containing float numbers in rows and columns (say 10 columns 50 rows) for further numerical process. Which format is best to save them in, eg, dictionary, list, or numpy array when it comes to performance? Will it be beneficial to convert all

Re: Recursion Performance Question

2008-07-24 Thread Anders J. Munch
B wrote: > > # pass in window handle and parent node > def gwl(node, hwnd): > if hwnd: > yield node, hwnd > for nd, wnd in Wnd.gwl(node.children[-1], GetWindow(hwnd, > GW_CHILD)): > yield nd, wnd > for nd, wnd in Wnd.gwl(node, GetWind

Re: Recursion Performance Question

2008-07-24 Thread Tim Golden
B wrote: Now it works, but it runs quite slow (compared to the c++ app). I changed gwl from strait recursion to use a generator and that helped, but it still takes 0.5-1.0 seconds to populate the tree. What I'm wondering is am I doing it in a really inefficient way, or is it just python? W

Recursion Performance Question

2008-07-23 Thread B
Hey I found some (VERY) old C++ code of mine that recursively built a tree of the desktop window handles (on windows) using: (they are stored in an STL vector) void FWL(HWND hwnd, int nFlag) // Recursive Function { hwnd = GetWindow(hwnd, nFlag); if(hwnd == NULL)

Re: Newbie Question: python mysqldb performance question

2007-05-20 Thread John Nagle
cjl wrote: > Group: > > I'm new to python and new to mysql. > > I have a csv file that is about 200,000 rows that I want to add to a > mysql database. Yes, I know that I can do this directly from the > mysql command line, but I am doing it through a python script so that > I can munge the data b

Re: Newbie Question: python mysqldb performance question

2007-05-20 Thread rurpy
On May 20, 5:55 pm, cjl <[EMAIL PROTECTED]> wrote: ...snip... > conn = MySQLdb.connect(db="database", host="localhost", user="root", > passwd="password") > c = conn.cursor() > > reader = csv.reader(open(sys.argv[1])) > for row in reader: > data1, data2, data3, data4 = row > data = (data1,da

Newbie Question: python mysqldb performance question

2007-05-20 Thread cjl
Group: I'm new to python and new to mysql. I have a csv file that is about 200,000 rows that I want to add to a mysql database. Yes, I know that I can do this directly from the mysql command line, but I am doing it through a python script so that I can munge the data before adding it. I have th

Re: performance question

2007-03-14 Thread Raymond Hettinger
[Eric Texier] > I need speed here. What will be the fastest method or does it matter? Follow Alex's advice and use the timeit module, but do not generalize from too small examples; otherwise, the relative timings will be thrown-off by issues like the time to lookup "write" and "a" and "str" (all of

Re: performance question

2007-03-14 Thread Alex Martelli
Eric Texier <[EMAIL PROTECTED]> wrote: > I need speed here. What will be the fastest method or does it matter? > > (for the example 'a' is only 3 values for the clarity of the example) > a = [1,3,4.] ## > > > method1: > > f.write("vec %f %f %f \n" % (a[0],a[1],a[2])) > > method2: > > f.write

performance question

2007-03-14 Thread Eric Texier
I need speed here. What will be the fastest method or does it matter? (for the example 'a' is only 3 values for the clarity of the example) a = [1,3,4.] ## method1: f.write("vec %f %f %f \n" % (a[0],a[1],a[2])) method2: f.write("vec " + str(a[0]) + " " + str(a[1]) + " " + str(a[2]) + "\n") a

Re: Specific performance question - Python vs. Java

2006-05-19 Thread Diez B. Roggisch
reinsn wrote: > Hi, > > I have got a specific question on performance: Is the overhead of > object creation in Python lower than in Java? I mean, I would argue, in > Java by object creation, the complete class is the model and all > methods and attributes are generated for the object. > In Python

Specific performance question - Python vs. Java

2006-05-19 Thread reinsn
Hi, I have got a specific question on performance: Is the overhead of object creation in Python lower than in Java? I mean, I would argue, in Java by object creation, the complete class is the model and all methods and attributes are generated for the object. In Python, methods and objects are onl