Re: how to format a python source file with tools?
On Nov 27, 9:58 am, "Diez B. Roggisch" wrote: [...] > > so i would like to have a tool to intelligently format the code for me > > and make the code more beautiful > > and automated. > > This is not possible. Consider the following situation: > [...] > Both are semantically radically different, and only you know which one > is the right one. > Diez I have to agree with Diez, there is no way to automate this. Some human intervention is needed. What I would like is an editor that will indicate what Python will consider a logical block (and sub-block, and sub-sub-block, etc.) It's complicated. I've tried to think of a way to do it, and have gotten lost after a few changes of indentation. Does anyone know of such a thing? I miss curly braces with an editor that will highlight matching parentheses, braces, etc. Gil -- http://mail.python.org/mailman/listinfo/python-list
Re: Most efficient way to "pre-grow" a list?
On Nov 6, 8:46 pm, gil_johnson wrote:
> >>> arr[0] = initializer
> >>> for i in range N:
> >>> arr.extend(arr)
>
> This doubles the array every time through the loop, and you can add
> the powers of 2 to get the desired result.
> Gil
To all who asked about my previous post,
I'm sorry I posted such a cryptic note before, I should have waited
until I
had the code available.
I am still new to Python, and I couldn't find a way to create an array
of
size N, with each member initialized to a given value. If there is
one,
please let me know.
I think Paul Rubin and Paul Rudin are right, if they really are 2
different
people, an array is a better solution if you're dealing with integers.
They
both mention numpy, which I know nothing about, or the array module.
kj, what *are* you going to do with this list/array?
As others have pointed out, there are differences between lists,
arrays, and
dictionaries.
The problem I was solving was this: I wanted an array of 32-bit
integers to
be used as a bit array, and I wanted it initialized with all bits set,
that
is, each member of the array had to be set to 4294967295. Of course,
you
could set your initializer to 0, or any other 32-bit number.
Originally I found that the doubling method I wrote about before was a
LOT
faster than appending the elements one at a time, and tonight I tried
the
"list = initializer * N" method. Running the code below, the doubling
method is still fastest, at least on my system.
Of course, as long as you avoid the 'one at a time' method, we're
talking
about fractions of a second, even for arrays that I think are huge,
like
the 536,870,912 byte beastie below.
[code]
# Written in Python 3.x
import array
import time
#* * * * * * * * * * * * * * * * * * * * * *
# Doubling method, run time = 0.413938045502
t0 = time.time()
newArray = array.array('I') # 32-bit unsigned
integers
newArray.append(4294967295)
for i in range(27):# 2**27 integers, 2**29
bytes
newArray.extend(newArray)
print(time.time() - t0)
print(newArray[134217727]) # confirm array is fully
initialized
#* * * * * * * * * * * * * * * * * * * * * *
# One at a time, run time = 28.5479729176
t0 = time.time()
newArray2 = array.array('I')
for i in range(134217728):# the same size as
above
newArray2.append(4294967295)
print(time.time() - t0)
print(newArray2[134217727]) # confirm array
#* * * * * * * * * * * * * * * * * * * * * *
# List with "*", run time = 1.06160402298
t0 = time.time()
newList = [4294967295] * 134217728
print(time.time() - t0)
print(newList[134217727]) # confirm list
[/code]
If, instead of 134,217,728 integers, I want something different, like
100,000,000, the method I use is:
[code]
#* * * * * * * * * * * * * * * * * * * * * *
# Not a power of 2, run time = 0.752086162567
t0 = time.time()
newArray = array.array('I')
tempArray = array.array('I')
tempArray.append(4294967295)
size = 1
while size: # chew through
'size' until it's gone
if (size & 1):# if this bit of
'size' is 1
newArray.extend(tempArray)# add a copy of the temp array
size >>= 1 # chew off one bit
tempArray.extend(tempArray) # double the size of the temp
array
print(time.time() - t0)
print(newArray[])
#* * * * * * * * * * * * * * * * * * * * * *
# # Not a power of 2, list with "*", run time = 1.19271993637
t0 = time.time()
newList = [4294967295] * 1
print(time.time() - t0)
print(newList[])
[/code]
I think it is interesting that the shorter list takes longer than the
one
that is a power of 2 in length. I think this may say that the "list =
initializer * N" method uses something similar to the doubling method.
Also, tempArray (above) gets reallocated frequently, and demonstrates
that reallocation is not a big problem.
Finally, I just looked into calling C functions, and found
PyMem_Malloc,
PyMem_Realloc, PyMem_Free, etc. in the Memory Management section of
the
Python/C API Reference Manual. This gives you uninitialized memory,
and
should be really fast, but it's 6:45 AM here, and I don't have the
energy
to try it.
Gil
--
http://mail.python.org/mailman/listinfo/python-list
Re: Most efficient way to "pre-grow" a list?
On Nov 6, 8:46 pm, gil_johnson wrote:
> I don't have the code with me, but for huge arrays, I have used
> something like:
>
> >>> arr[0] = initializer
> >>> for i in range N:
> >>> arr.extend(arr)
>
> This doubles the array every time through the loop, and you can add
> the powers of 2 to get the desired result.
> Gil
To all who asked about my previous post,
I'm sorry I posted such a cryptic note before, I should have waited
until I had the code available.
I am still new to Python, and I couldn't find a way to create an array
of size N, with each member initialized to a given value. If there is
one, please let me know.
I think Paul Rubin and Paul Rudin are right, if they really are 2
different people, an array is a better solution if you're dealing with
integers. They both mention numpy, which I know nothing about, or the
array module.
kj, what *are* you going to do with this list/array? As others have
pointed out, there are differences between lists, arrays, and
dictionaries.
The problem I was solving was this: I wanted an array of 32-bit
integers to be used as a bit array, and I wanted it initialized with
all bits set, that is, each member of the array had to be set to
4294967295. Of course, you could set your initializer to 0, or any
other 32-bit number.
Originally I found that the doubling method I wrote about before was a
LOT faster than appending the elements one at a time, and tonight I
tried the "list = initializer * N" method. Running the code below, the
doubling method is still fastest, at least on my system.
Of course, as long as you avoid the 'one at a time' method, we're
talking about fractions of a second, even for arrays that I think are
huge, like the 536,870,912 byte beastie below.
[code]
# Written in Python 3.x
import array
import time
#* * * * * * * * * * * * * * * * * * * * * *
# Doubling method, run time = 0.413938045502
t0 = time.time()
newArray = array.array('I') # 32-bit unsigned
integers
newArray.append(4294967295)
for i in range(27): # 2**27 integers, 2**29 bytes
newArray.extend(newArray)
print(time.time() - t0)
print(newArray[134217727]) # confirm array is fully initialized
#* * * * * * * * * * * * * * * * * * * * * *
# One at a time, run time = 28.5479729176
t0 = time.time()
newArray2 = array.array('I')
for i in range(134217728):# the same size as above
newArray2.append(4294967295)
print(time.time() - t0)
print(newArray2[134217727]) # confirm array
#* * * * * * * * * * * * * * * * * * * * * *
# List with "*", run time = 1.06160402298
t0 = time.time()
newList = [4294967295] * 134217728
print(time.time() - t0)
print(newList[134217727]) # confirm list
[/code]
If, instead of 134,217,728 integers, I want something different, like
100,000,000, the method I use is:
[code]
#* * * * * * * * * * * * * * * * * * * * * *
# Not a power of 2, run time = 0.752086162567
t0 = time.time()
newArray = array.array('I')
tempArray = array.array('I')
tempArray.append(4294967295)
size = 1
while size: # chew through
'size' until it's gone
if (size & 1):# if this bit of
'size' is 1
newArray.extend(tempArray)# add a copy of the temp array
size >>= 1# chew off one bit
tempArray.extend(tempArray) # double the size of the temp
array
print(time.time() - t0)
print(newArray[])
#* * * * * * * * * * * * * * * * * * * * * *
# # Not a power of 2, list with "*", run time = 1.19271993637
t0 = time.time()
newList = [4294967295] * 1
print(time.time() - t0)
print(newList[])
[/code]
I think it is interesting that the shorter list takes longer than the
one that is a power of 2 in length. I think this may say that the
"list = initializer * N" method uses something similar to the doubling
method.
Also, tempArray (above) gets reallocated frequently, and demonstrates
that reallocation is not a big problem.
Finally, I just looked into calling C functions, and found
PyMem_Malloc, PyMem_Realloc, PyMem_Free, etc. in the Memory Management
section of the Python/C API Reference Manual. This gives you
uninitialized memory, and should be really fast, but it's 6:45 AM
here, and I don't have the energy to try it.
Gil
--
http://mail.python.org/mailman/listinfo/python-list
Re: Most efficient way to "pre-grow" a list?
On Nov 9, 10:56 pm, "Gabriel Genellina" wrote:
>
> [much cutting]
>
> def method3a():
> newArray = array.array('I', [INITIAL_VALUE]) * SIZE
> assert len(newArray)==SIZE
> assert newArray[SIZE-1]==INITIAL_VALUE
>
> [more cutting]
>
> So arrays are faster than lists, and in both cases one_item*N outperforms
> your doubling algorithm.
> Adding one item at a time is -at least- several hundred times slower; I
> was not patient enough to wait.
>
> > Finally, I just looked into calling C functions, and found
> > PyMem_Malloc, PyMem_Realloc, PyMem_Free, etc. in the Memory Management
> > section of the Python/C API Reference Manual. This gives you
> > uninitialized memory, and should be really fast, but it's 6:45 AM
> > here, and I don't have the energy to try it.
>
> No, those are for internal use only, you can't use the resulting pointer
> in Python code. An array object is a contiguous memory block, so you don't
> miss anything.
>
Gabriel,
Thanks for the reply - your method 3a was what I wanted,
I missed "array.array('I', [INITIAL_VALUE]) * SIZ" on my
earlier pass through the Python documentation. I included
the 'one at a time' method because that was my first shot
at the problem - I was so happy to find a method that would
get the job done today that I didn't look farther than the
doubling method.
Yes, I know that I was comparing lists and arrays. Part of the
confusion in this thread is that the problem hasn't been
defined very well. The original post asked about arrays
but the solution proposed generated a list. At this point,
I have learned a lot about arrays, lists and dictionaries, and
will be better able to chose the right solution to future
problems.
It's too bad about the C functions, they sure sounded good.
And with that, I think I will withdraw from the field, and go
write some code and learn about timeit.
Gil
--
http://mail.python.org/mailman/listinfo/python-list
Re: A "terminators' club" for clp
On Nov 13, 5:29 pm, kj wrote:
[...]
> Or it could be set up so that at least n > 1 "delete" votes and no
> "keep" votes are required to get something nixed. Etc.
>
> This seems simpler than all-out moderation.
>
> ("all-out moderation"? now, there's an oxymoron for ya!)
>
How about using a "rank this post" feature? Anybody could rank a post
as spam, and a sufficiently large number of negatives would quickly
draw the attention of someone with the power to kill the message. I
suppose even this is subject to abuse, allowing harassment of a
legitimate poster., but my guess is that the votes against counterfeit
Nike shoes, etc., would outnumber the most energetic "vote troll."
Gil
--
http://mail.python.org/mailman/listinfo/python-list
Re: A "terminators' club" for clp
On Nov 14, 12:08 pm, r wrote: > On Nov 14, 7:28 am, gil_johnson wrote: > Actually there is a "rank this post" (gotta be careful with that > verbage!) AND a "report this post as spam". Of course it only exists > in GG's and not Usenet. I *do* know that the star system is used quite > frequently, but i doubt anyone bothers to use the "report as spam" > link since it seems have no effect whatsoever. Heh. I should look around more before posting. It does prove your point, though. The 'spam' button is ubiquitous, but so seldom used it's forgotten. Actually, my enthusiasm for my idea faded with a little thought. It is an extra effort to open spam to get to the 'spam' button, and sends the message to the spammer that people are, indeed opening their junk. I'd use it if I opened a message with a deceptive subject. Gil -- http://mail.python.org/mailman/listinfo/python-list
