Re: sync databse table based on current directory data without losign previous values
Νίκος Γκρ33κ writes:
> How can i update the databse to only contain the existing filenames without
> losing the previous stored data?
Basically you need to keep a list (or better, a set) containing all
current filenames that you are going to insert, and finally do another
"inverse" loop where you scan all the records and delete those that are
not present anymore.
Of course, this assume you have a "bidirectional" identity between the
filenames you are loading and the records you are inserting, which is
not the case in the code you show:
> #read the containing folder and insert new filenames
> for result in os.walk(path):
> for filename in result[2]:
Here "filename" is just that, not the full path: this could result in
collisions, if your are actually loading a *tree* instead of a flat
directory, that is multiple source files are squeezed into a single
record in your database (imagine "/foo/index.html" and
"/foo/subdir/index.html").
With that in mind, I would do something like the following:
# Compute a set of current fullpaths
current_fullpaths = set()
for root, dirs, files in os.walk(path):
for fullpath in files:
current_fullpaths.add(os.path.join(root, file))
# Load'em
for fullpath in current_fullpaths:
try:
#find the needed counter for the page URL
cur.execute('''SELECT URL FROM files WHERE URL = %s''', (fullpath,) )
data = cur.fetchone()#URL is unique, so should only be one
if not data:
#first time for file; primary key is automatic, hit is defaulted
cur.execute('''INSERT INTO files (URL, host, lastvisit) VALUES (%s, %s,
%s)''', (fullpath, host, date) )
except MySQLdb.Error, e:
print ( "Query Error: ", sys.exc_info()[1].excepinfo()[2] )
# Delete spurious
cur.execute('''SELECT url FROM files''')
for rec in cur:
fullpath = rec[0]
if fullpath not in current_fullpaths:
other_cur.execute('''DELETE FROM files WHERE url = %s''', (fullpath,))
Of course here I am assuming a lot (a typical thing we do to answer your
questions :-), in particular that the "url" field content matches the
filesystem layout, which may not be the case. Adapt it to your usecase.
hope this helps,
ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it | -- Fortunato Depero, 1929.
--
http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
Τη Τετάρτη, 6 Μαρτίου 2013 10:19:06 π.μ. UTC+2, ο χρήστης Lele Gaifax έγραψε:
> Νίκος Γκρ33κ writes:
>
>
>
> > How can i update the databse to only contain the existing filenames
> > without losing the previous stored data?
>
>
>
> Basically you need to keep a list (or better, a set) containing all
>
> current filenames that you are going to insert, and finally do another
>
> "inverse" loop where you scan all the records and delete those that are
>
> not present anymore.
>
>
>
> Of course, this assume you have a "bidirectional" identity between the
>
> filenames you are loading and the records you are inserting, which is
>
> not the case in the code you show:
>
>
>
> > #read the containing folder and insert new filenames
>
> > for result in os.walk(path):
>
> > for filename in result[2]:
>
>
>
> Here "filename" is just that, not the full path: this could result in
>
> collisions, if your are actually loading a *tree* instead of a flat
>
> directory, that is multiple source files are squeezed into a single
>
> record in your database (imagine "/foo/index.html" and
>
> "/foo/subdir/index.html").
>
>
>
> With that in mind, I would do something like the following:
>
>
>
> # Compute a set of current fullpaths
>
> current_fullpaths = set()
>
> for root, dirs, files in os.walk(path):
>
> for fullpath in files:
>
> current_fullpaths.add(os.path.join(root, file))
>
>
>
> # Load'em
>
> for fullpath in current_fullpaths:
>
>
>
> try:
>
> #find the needed counter for the page URL
>
> cur.execute('''SELECT URL FROM files WHERE URL = %s''', (fullpath,) )
>
> data = cur.fetchone()#URL is unique, so should only be one
>
>
>
> if not data:
>
> #first time for file; primary key is automatic, hit is defaulted
>
> cur.execute('''INSERT INTO files (URL, host, lastvisit) VALUES (%s,
> %s, %s)''', (fullpath, host, date) )
>
> except MySQLdb.Error, e:
>
> print ( "Query Error: ", sys.exc_info()[1].excepinfo()[2] )
>
>
>
> # Delete spurious
>
> cur.execute('''SELECT url FROM files''')
>
> for rec in cur:
>
> fullpath = rec[0]
>
> if fullpath not in current_fullpaths:
>
> other_cur.execute('''DELETE FROM files WHERE url = %s''', (fullpath,))
>
>
>
> Of course here I am assuming a lot (a typical thing we do to answer your
>
> questions :-), in particular that the "url" field content matches the
>
> filesystem layout, which may not be the case. Adapt it to your usecase.
>
>
>
> hope this helps,
>
> ciao, lele.
>
> --
>
> nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
>
> real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
>
> l...@metapensiero.it | -- Fortunato Depero, 1929.
You are fantastic! Your straightforward logic amazes me!
Thank you very much for making things clear to me!!
But there is a slight problem when iam trying to run the code iam presenting
this error ehre you can see its output here:
http://superhost.gr/cgi-bin/files.py
--
http://mail.python.org/mailman/listinfo/python-list
Re: Inserting-embedding some html data at the end of a .py file
Hi,
On Tue, Mar 05, 2013 at 09:39:19AM -0800, Νίκος Γκρ33κ wrote:
> But i did, I just tried this:
>
> # open html template
> if htmlpage.endswith('.html'):
> f = open( "/home/nikos/public_html/" + htmlpage )
>
> htmldata = f.read()
> counter = ''' href="mailto:supp...@superhost.gr";>
>cellpadding=2 bgcolor=black>
>color=lime>Αριθμός Επισκεπτών
>href="http://superhost.gr/?show=stats";> %d
> ''' % data[0]
> else:
> f = open( "/home/nikos/public_html/cgi-bin/" + htmlpage )
>
> htmldata = f.read()
> counter = '''
> print ''' href="mailto:supp...@superhost.gr";>
>cellpadding=2 bgcolor=black>
>color=lime>Αριθμός Επισκεπτών
>href="http://superhost.gr/?show=stats";> %d
> '''
> ''' % data[0]
>
> template = htmldata + counter
> print ( template )
> =
>
> But still doens't embed correctly the additional html data at the end of the
> .py files.
>
> Do you have an idea?
as someone said: You're doing it the wrong way.
I would recommend to use a template-engine; then you can put the
complete html-design (and some design-control-structures) into
the template (and *not* into the cgi) and fill data into the
template with a python-script.
Roland
--
http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
Νίκος Γκρ33κ writes: > Thank you very much for making things clear to me!! You're welcome, even more if you spend 1 second to trim your answers removing unneeded citation :-) > > But there is a slight problem when iam trying to run the code iam presenting > this error ehre you can see its output here: > > http://superhost.gr/cgi-bin/files.py Sorry, this seems completely unrelated, and from the little snippet that appear on that page I cannot understand what's going on there. ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929. -- http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
Its about the following line of code:
current_fullpaths.add( os.path.join(root, files) )
that presents the following error:
: 'list' object has no attribute 'startswith'
args = ("'list' object has no attribute 'startswith'",)
message = "'list' object has no attribute 'startswith'"
join calls some module that find difficulty when parsing its line:
/usr/lib64/python2.6/posixpath.py in
join(a='/home/nikos/public_html/data/files/',
*p=(['\xce\x9a\xcf\x8d\xcf\x81\xce\xb9\xce\xb5
\xce\x99\xce\xb7\xcf\x83\xce\xbf\xcf\x8d
\xce\xa7\xcf\x81\xce\xb9\xcf\x83\xcf\x84\xce\xad
\xce\x95\xce\xbb\xce\xad\xce\xb7\xcf\x83\xce\xbf\xce\xbd \xce\x9c\xce\xb5.mp3',
'\xce\xa0\xce\xb5\xcf\x81\xce\xaf \xcf\x84\xcf\x89\xce\xbd
\xce\x9b\xce\xbf\xce\xb3\xce\xb9\xcf\x83\xce\xbc\xcf\x8e\xce\xbd.mp3'],))
63 path = a
64 for b in p:
65 if b.startswith('/'):
--
http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
Perhaps because my filenames is in greek letters that thsi error is presented but i'am not sure. Maybe we can join root+files and store it to the set() someway differenyl -- http://mail.python.org/mailman/listinfo/python-list
Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
Hello there, I am using python 2.7.1 built on HP-11.23 a Itanium 64 bit box. I discovered following behavior whereby the python process doesn't seem to release memory utilized even after a variable is set to None, and "deleted". I use glance tool to monitor the memory utilized by this process. Obviously after the for loop is executed, the memory used by this process has hiked to a few MB. However, after "del" is executed to both I and str variables, the memory of that process still stays at where it was. Any idea why? >>> for i in range(10L): ... str=str+"%s"%(i,) ... >>> i=None >>> str=None >>> del i >>> del str -- http://mail.python.org/mailman/listinfo/python-list
Re: book advice
On Mar 2, 1:59 am, leonardo selmi wrote: > hi > > is there anyone can suggest me a good book to learn python? i read many but > there is always > something unclear or examples which give me errors. The following written assuming you are as new to programming generally as to python specifically. If not then please excuse me. it is written based on this line: > how can I start building a sound educational background It is completely natural that you assume learning programming means learning a programming language. And in that context python is as good a choice as you could make. However the assumption is wrong-headed. Sure you cannot program without using some programming language. Yet knowing a programming language does not do much by way of knowing programming. Look up any of the online resources on computer science and you will find courses/material on a variety of stuff such as: - operating systems - compilers - data structures - algorithms - a variety of almost unrelated flavors of math (eg graph theory, discrete math, numerical analysis) - a variety of almost unrelated flavors of theory (eg semantics, automata theory etc) - then all sorts of 'modern' stuff eg web/security/cloud etc Somewhere in all this you would find a course on programming languages and an intro course on programming -- probably using python or some other language. Now I will admit that this 'classical' approach is often misguided and has wrong emphasis. I have a series of blog posts starting http://blog.languager.org/2011/02/cs-education-is-fat-and-weak-1.html on the mess in CS education. However the mess in CS education notwithstanding, the fact does not change that for a 'sound educational background' you should be spending 1/5 your time on a specific programming technology such as python and the remaining on more generic stuff such as Ive listed above -- http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
On Wednesday, March 6, 2013 9:43:34 AM UTC, Νίκος Γκρ33κ wrote:
> Perhaps because my filenames is in greek letters that thsi error is presented
> but i'am not sure.
>
>
>
> Maybe we can join root+files and store it to the set() someway differenyl
well, the error refers to the line "if b.startswith('/'): " and states "'list'
object has no attribute 'startswith'"
so b is assigned to a list type and list does not have a 'startswith' method or
attribute.
I Thought .startswith() was a string method but if it's your own method then I
apologize (though if it is, I personally would have made a class that inherited
from list rather than adding it to list itself)
can you show where you are assigning b (or if its meant to be a list or string
object)
--
http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
Νίκος Γκρ33κ writes: > Its about the following line of code: > > current_fullpaths.add( os.path.join(root, files) ) I'm sorry, typo on my part. That should have been "fullpath", not "file" (and neither "files" as you wrongly reported back!): # Compute a set of current fullpaths current_fullpaths = set() for root, dirs, files in os.walk(path): for fullpath in files: current_fullpaths.add(os.path.join(root, fullpath)) ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929. -- http://mail.python.org/mailman/listinfo/python-list
Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
On Wednesday, March 6, 2013 10:11:12 AM UTC, Wong Wah Meng-R32813 wrote: > Hello there, > > > > I am using python 2.7.1 built on HP-11.23 a Itanium 64 bit box. > > > > I discovered following behavior whereby the python process doesn't seem to > release memory utilized even after a variable is set to None, and "deleted". > I use glance tool to monitor the memory utilized by this process. Obviously > after the for loop is executed, the memory used by this process has hiked to > a few MB. However, after "del" is executed to both I and str variables, the > memory of that process still stays at where it was. > > > > Any idea why? > > > > >>> for i in range(10L): > > ... str=str+"%s"%(i,) > > ... > > >>> i=None > > >>> str=None > > >>> del i > > >>> del str Hi, I'm new here so I'm making mistakes too but I know they don't like it when you ask your question in someone else's question. that being said, to answer your question: Python uses a 'garbage collector'. When you delete something, all references are removed from the object in memory, the memory itself will not be freed until the next time the garbage collector runs. When that happens, all objects without references in memory are removed and the memory freed. If you wait a while you should see that memory free itself. -- http://mail.python.org/mailman/listinfo/python-list
Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
On 3/6/2013 5:11 AM, Wong Wah Meng-R32813 wrote: Hello there, I am using python 2.7.1 built on HP-11.23 a Itanium 64 bit box. I discovered following behavior whereby the python process doesn't seem to release memory utilized even after a variable is set to None, and "deleted". I use glance tool to monitor the memory utilized by this process. Obviously after the for loop is executed, the memory used by this process has hiked to a few MB. However, after "del" is executed to both I and str variables, the memory of that process still stays at where it was. Whether memory freed by deleting an object is returned to and taken by the OS depends on the OS and other factors like like the size and layout of the freed memory, probably the history of memory use, and for CPython, the C compiler's malloc/free implementation. At various times, the Python memory handlers have been rewritten to encourage/facilitate memory return, but Python cannot control the process. for i in range(10L): str=str+"%s"%(i,) i=None; str=None # not necessary > del i; del str Reusing built-in names for unrelated purposes is generally a bad idea, although the final deletion does restore access to the builtin. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
On 06/03/2013 07:45, Νίκος Γκρ33κ wrote:
I'am using this snipper to read a current directory and insert all filenames
into a databse and then display them.
But what happens when files are get removed form the directory?
The inserted records into databse remain.
How can i update the databse to only contain the existing filenames without
losing the previous stored data?
Here is what i ahve so far:
==
path = "/home/nikos/public_html/data/files/"
#read the containing folder and insert new filenames
for result in os.walk(path):
You were told yesterday at least twice that os.walk returns a tuple but
you still insist on refusing to take any notice of our replies when it
suits you, preferring instead to waste everbody's time with these
questions. Or are you trying to get into the Guinness Book of World
Records for the laziest bastard on the planet?
for filename in result[2]:
try:
#find the needed counter for the page URL
cur.execute('''SELECT URL FROM files WHERE URL = %s''',
(filename,) )
data = cur.fetchone()#URL is unique, so should
only be one
if not data:
#first time for file; primary key is automatic,
hit is defaulted
cur.execute('''INSERT INTO files (URL, host,
lastvisit) VALUES (%s, %s, %s)''', (filename, host, date) )
except MySQLdb.Error, e:
print ( "Query Error: ",
sys.exc_info()[1].excepinfo()[2] )
==
Thank you.
--
Cheers.
Mark Lawrence
--
http://mail.python.org/mailman/listinfo/python-list
RE: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
Apologies as after I have left the group for a while I have forgotten how not to post a question on top of another question. Very sorry and appreciate your replies. I tried explicitly calling gc.collect() and didn't manage to see the memory footprint reduced. I probably haven't left the process idle long enough to see the internal garbage collection takes place but I will leave it idle for more than 8 hours and check again. Thanks! -Original Message- From: Python-list [mailto:python-list-bounces+wahmeng=freescale@python.org] On Behalf Of Bryan Devaney Sent: Wednesday, March 06, 2013 6:25 PM To: python-list@python.org Cc: python-list@python.org Subject: Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) On Wednesday, March 6, 2013 10:11:12 AM UTC, Wong Wah Meng-R32813 wrote: > Hello there, > > > > I am using python 2.7.1 built on HP-11.23 a Itanium 64 bit box. > > > > I discovered following behavior whereby the python process doesn't seem to > release memory utilized even after a variable is set to None, and "deleted". > I use glance tool to monitor the memory utilized by this process. Obviously > after the for loop is executed, the memory used by this process has hiked to > a few MB. However, after "del" is executed to both I and str variables, the > memory of that process still stays at where it was. > > > > Any idea why? > > > > >>> for i in range(10L): > > ... str=str+"%s"%(i,) > > ... > > >>> i=None > > >>> str=None > > >>> del i > > >>> del str Hi, I'm new here so I'm making mistakes too but I know they don't like it when you ask your question in someone else's question. that being said, to answer your question: Python uses a 'garbage collector'. When you delete something, all references are removed from the object in memory, the memory itself will not be freed until the next time the garbage collector runs. When that happens, all objects without references in memory are removed and the memory freed. If you wait a while you should see that memory free itself. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
RE: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
Thanks for youre reply. I built python 2.7.1 binary myself on the HP box and I wasn't aware there is any configuration or setup that I need to modify in order to activate or engage the garbage collection (or even setting the memory size used). Probably you are right it leaves it to the OS itself (in this case HP-UX) to clean it up as after python removes the reference to the address of the variables the OS still thinks the python process should still owns it until the process exits. Regards, Wah Meng -Original Message- From: Python-list [mailto:python-list-bounces+wahmeng=freescale@python.org] On Behalf Of Terry Reedy Sent: Wednesday, March 06, 2013 7:00 PM To: python-list@python.org Subject: Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) On 3/6/2013 5:11 AM, Wong Wah Meng-R32813 wrote: > Hello there, > > I am using python 2.7.1 built on HP-11.23 a Itanium 64 bit box. > > I discovered following behavior whereby the python process doesn't > seem to release memory utilized even after a variable is set to None, > and "deleted". I use glance tool to monitor the memory utilized by > this process. Obviously after the for loop is executed, the memory > used by this process has hiked to a few MB. However, after "del" is > executed to both I and str variables, the memory of that process still > stays at where it was. Whether memory freed by deleting an object is returned to and taken by the OS depends on the OS and other factors like like the size and layout of the freed memory, probably the history of memory use, and for CPython, the C compiler's malloc/free implementation. At various times, the Python memory handlers have been rewritten to encourage/facilitate memory return, but Python cannot control the process. > for i in range(10L): > str=str+"%s"%(i,) > i=None; str=None # not necessary > del i; del str Reusing built-in names for unrelated purposes is generally a bad idea, although the final deletion does restore access to the builtin. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Difference in RE between 3.2 and 3.3 (or Aaron Swartz memorial)
On 2013-02-26, 16:25 GMT, Terry Reedy wrote: > On 2/21/2013 4:22 PM, Matej Cepl wrote: >> as my method to commemorate Aaron Swartz, I have decided to port his >> html2text to work fully with the latest python 3.3. After some time >> dealing with various bugs, I have now in my repo >> https://github.com/mcepl/html2text (branch python3) working solution >> which works all the way to python 3.2 (inclusive; >> https://travis-ci.org/mcepl/html2text). However, the last problem >> remains. This >> >> Run this command: >> ls -l *.html >> ? >> >> should lead to >> >>* Run this command: >> >> ls -l *.html >> >>* ? >> >> but it doesn’t. It leads to this (with python 3.3 only) >> >> * Run this command: >>ls -l *.html >> >> * ? >> >> Does anybody know about something which changed in modules re or >> http://docs.python.org/3.3/whatsnew/changelog.html between 3.2 and >> 3.3, which could influence this script? > > Search the changelob or 3.3 misc/News for items affecting those two > modules. There are at least 4. > http://docs.python.org/3.3/whatsnew/changelog.html > > It is faintly possible that the switch from narrow/wide builds to > unified builds somehow affected that. Have you tested with 2.7/3.2 on > both narrow and wide unicode builds? So, in the end, I have went the long way and bisected cpython to find the commit which broke my tests, and it seems that the culprit is http://hg.python.org/cpython/rev/123f2dc08b3e so it is clearly something Unicode related. Unfortunately, it really doesn't tell me what exactly is broken (is it a known regression) and if there is known workaround. Could anybody suggest a way how to find bugs on http://bugs.python.org related to some particular commit (plain search for 123f2dc0 didn’t find anything). Any thoughts? Matěj P.S.: Crossposting to python-devel in hope there would be somebody understanding more about that particular commit. For that I have also intentionally not trim the original messages to preserve context. -- http://mail.python.org/mailman/listinfo/python-list
Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
On 03/06/2013 05:25 AM, Bryan Devaney wrote: On Wednesday, March 6, 2013 10:11:12 AM UTC, Wong Wah Meng-R32813 wrote: Hello there, I am using python 2.7.1 built on HP-11.23 a Itanium 64 bit box. I discovered following behavior whereby the python process doesn't seem to release memory utilized even after a variable is set to None, and "deleted". I use glance tool to monitor the memory utilized by this process. Obviously after the for loop is executed, the memory used by this process has hiked to a few MB. However, after "del" is executed to both I and str variables, the memory of that process still stays at where it was. Python uses a 'garbage collector'. When you delete something, all references are removed from the object in memory, the memory itself will not be freed until the next time the garbage collector runs. When that happens, all objects without references in memory are removed and the memory freed. If you wait a while you should see that memory free itself. Actually, no. The problem with monitoring memory usage from outside the process is that memory "ownership" is hierarchical, and each hierarchy deals in bigger chunks. So when the CPython runtime calls free() on a particular piece of memory, the C runtime may or may not actually release the memory for use by other processes. Since the C runtime grabs big pieces from the OS, and parcels out little pieces to CPython, a particular big piece can only be freed if ALL the little pieces are free. And even then, it may or may not choose to do so. Completely separate from that are the two mechanisms that CPython uses to free its pieces. It does reference counting, and it does garbage collecting. In this case, only the reference counting is relevant, as when it's done there's no garbage left to collect. When an object is no longer referenced by anything, its count will be zero, and it will be freed by calling the C library function. GC is only interesting when there are cycles in the references, such as when a list contains as one of its elements a tuple, which in turn contains the original list. Sound silly? No, it's quite common once complex objects are created which reference each other. The counts don't go to zero, and the objects wait for garbage collection. OP: There's no need to set to None and also to del the name. Since there's only one None object, keeping another named reference to that object has very little cost. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
On 03/06/2013 07:31 AM, Wong Wah Meng-R32813 wrote: Apologies as after I have left the group for a while I have forgotten how not to post a question on top of another question. Very sorry and appreciate your replies. I tried explicitly calling gc.collect() and didn't manage to see the memory footprint reduced. I probably haven't left the process idle long enough to see the internal garbage collection takes place but I will leave it idle for more than 8 hours and check again. Thanks! You're top-posting, which makes things very confusing, since your contribution to the message is out of accepted order. Put your remarks after the part you're commenting on, and delete anything following your message, as it clearly didn't need your comments. Once you've called gc.collect(), there's no point in waiting 8 hours for it to run again. It either triggered the C runtime's logic or it didn't, and running it again won't help unless in the meantime you rearranged the remaining allocated blocks. Accept the fact that not all freeing of memory blocks can possibly make it through to the OS. If they did, we'd have a minimum object size of at least 4k on the Pentium, and larger on some other processors. We'd also have performance that would crawl. So an external tool can only give you a very approximate size for what's going on in your own code. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
On 03/06/2013 05:27 AM, Lele Gaifax wrote: Νίκος Γκρ33κ writes: Its about the following line of code: current_fullpaths.add( os.path.join(root, files) ) I'm sorry, typo on my part. That should have been "fullpath", not "file" (and neither "files" as you wrongly reported back!): # Compute a set of current fullpaths current_fullpaths = set() for root, dirs, files in os.walk(path): for fullpath in files: 'fullpath' is a rather misleading name to use, since the 'files' list contains only the terminal node of the file name. It's only a full path after you do the following join. current_fullpaths.add(os.path.join(root, fullpath)) -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
On Wed, Mar 6, 2013 at 10:52 PM, Mark Lawrence wrote: > On 06/03/2013 07:45, Νίκος Γκρ33κ wrote: >> blah blah blah >> blah blah blah >> blah blah blah > You were told yesterday at least twice that os.walk returns a tuple but you > still insist on refusing to take any notice of our replies when it suits > you, preferring instead to waste everbody's time with these questions. Or > are you trying to get into the Guinness Book of World Records for the > laziest bastard on the planet? This is the same person who posts under the name Ferrous Cranus. His threads are open-ended time sinks; he generally expects everyone else to do the work, while his contribution is to complain that the provided code doesn't fit some arbitrary set of restrictions that don't always even make sense. Guinness might be interested in him as "Most Successful Troll", though. He certainly absorbed a lot of my dev time before I gave up on him, and by the look of things, he's still getting other people to do his work for him. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
On Wed, 06 Mar 2013 12:52:00 +0100, Mark Lawrence wrote: On 06/03/2013 07:45, Νίκος Γκρ33κ wrote: I'am using this snipper to read a current directory and insert all filenames into a databse and then display them. But what happens when files are get removed form the directory? The inserted records into databse remain. How can i update the databse to only contain the existing filenames without losing the previous stored data? Here is what i ahve so far: == path = "/home/nikos/public_html/data/files/" #read the containing folder and insert new filenames for result in os.walk(path): You were told yesterday at least twice that os.walk returns a tuple but you still insist on refusing to take any notice of our replies when it suits you, preferring instead to waste everbody's time with these questions. Or are you trying to get into the Guinness Book of World Records for the laziest bastard on the planet? Hold on a sec ... He has for result in os.walk(path): for filename in result[2]: So he *did* take notice of that. Nikos: Expectation is to iterate through a tuple like this: for dirpath, dirnames, filenames in os.walk(path): ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Controlling number of zeros of exponent in scientific notation
In article , fa...@squashclub.org wrote: > Instead of: > > 1.8e-04 > > I need: > > 1.8e-004 > > So two zeros before the 4, instead of the default 1. Just out of curiosity, what's the use case here? -- http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
Dave Angel writes: >># Compute a set of current fullpaths >>current_fullpaths = set() >>for root, dirs, files in os.walk(path): >> for fullpath in files: > > 'fullpath' is a rather misleading name to use, since the 'files' list > contains only the terminal node of the file name. It's only a full > path after you do the following join. Yes, you're right. Dunno what urged me to ``M-x replace-string file fullpath`` introducing both an error and a bad variable name, most probably the unconscious desire of not clobbering builtin names... :-) Thanks for pointing it out, ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929. -- http://mail.python.org/mailman/listinfo/python-list
Re: Controlling number of zeros of exponent in scientific notation
On 6 mar, 15:03, Roy Smith wrote: > In article , > > fa...@squashclub.org wrote: > > Instead of: > > > 1.8e-04 > > > I need: > > > 1.8e-004 > > > So two zeros before the 4, instead of the default 1. > > Just out of curiosity, what's the use case here? -- >>> from vecmat6 import * >>> from svdecomp6 import * >>> from vmio6 import * >>> mm = NewMat(3, 2) >>> mm[0][0] = 1.0; mm[0][1] = 2.0e-178 >>> mm[1][0] = 3.0; mm[1][1] = 4.0e-1428 >>> mm[2][0] = 5.0; mm[2][1] = 6.0 >>> pr(mm, 'mm =') mm = ( 1.0e+000 2.0e-178 ) ( 3.0e+000 0.0e+000 ) ( 5.0e+000 6.0e+000 ) >>> aa, vv, bbt = SVDecompFull(mm) >>> pr(aa, 'aa =') aa = ( 3.04128e-001 -8.66366e-002 ) ( 9.12385e-001 -2.59910e-001 ) ( -2.73969e-001 -9.61739e-001 ) >>> pr(bbt, 'bbt =') bbt = ( 7.12974e-001 -7.01190e-001 ) ( -7.01190e-001 -7.12974e-001 ) >>> rr = MatMulMatMulMat(aa, vv, bbt) >>> pr(rr, 'rr =') rr = ( 1.0e+000 -1.38778e-015 ) ( 3.0e+000 -4.44089e-016 ) ( 5.0e+000 6.0e+000 ) >>> jmf -- http://mail.python.org/mailman/listinfo/python-list
Creating an object that can track when its attributes are modified
I am trying to make an object that can track when its attributes have been assigned new values, and which can rollback to previous values where necessary. I have the following code which I believe works, but would like to know if there are simpler ways to achieve this goal, or if there are any bugs I haven't seen yet. class ChangeTrackingObject(object): def __init__(self): self.clean() def clean(self): """Mark all attributes as unmodified.""" object.__setattr__(self, '_dirty_attributes', dict()) def dirty_vals(self): """Returns all dirty values.""" return dict( [ (k,v) for k,v in self.__dict__.iteritems() if k in self._dirty_attributes] ) def get_changes_and_clean(self): """Helper that collects all the changes and returns them, cleaning the dirty flags at the same time.""" changes = self.dirty_vals() self.clean() return changes def rollback(self): """Reset attributes to their previous values.""" for k,v in self._dirty_attributes.iteritems(): object.__setattr__(self, k, v) self.clean() def __setattr__(self, key, value): # If the first modification to this attribute, store the old value if key not in self._dirty_attributes: if key in self.__dict__: self._dirty_attributes[key] = object.__getattribute__(self, key) else: self._dirty_attributes[key] = None # Set the new value object.__setattr__(self, key, value) I am aware that adding a new attribute and then calling rollback() leaves the new attribute in place with a None value - maybe I can use a special DeleteMe marker object in the _dirty_attributes dict along with a loop that calls delattr on any attribute that has that value after a rollback. I also believe that this won't catch modification to existing attributes as opposed to assignments: eg. if one of the attributes is a list and I append to it, this system won't notice. Is that something I can rectify easily? Any other comments or suggestions? Thanks, -- Ben Sizer -- http://mail.python.org/mailman/listinfo/python-list
Re: sync databse table based on current directory data without losign previous values
Τη Τετάρτη, 6 Μαρτίου 2013 4:04:26 μ.μ. UTC+2, ο χρήστης Michael Ross έγραψε: > On Wed, 06 Mar 2013 12:52:00 +0100, Mark Lawrence > > wrote: > > > > > On 06/03/2013 07:45, Νίκος Γκρ33κ wrote: > > >> I'am using this snipper to read a current directory and insert all > > >> filenames into a databse and then display them. > > >> > > >> But what happens when files are get removed form the directory? > > >> The inserted records into databse remain. > > >> How can i update the databse to only contain the existing filenames > > >> without losing the previous stored data? > > >> > > >> Here is what i ahve so far: > > >> > > >> == > > >> path = "/home/nikos/public_html/data/files/" > > >> > > >> #read the containing folder and insert new filenames > > >> for result in os.walk(path): > > > > > > You were told yesterday at least twice that os.walk returns a tuple but > > > you still insist on refusing to take any notice of our replies when it > > > suits you, preferring instead to waste everbody's time with these > > > questions. Or are you trying to get into the Guinness Book of World > > > Records for the laziest bastard on the planet? > > > > Hold on a sec ... > > > > He has > > > > for result in os.walk(path): > > for filename in result[2]: > > > > So he *did* take notice of that. > > > > > > Nikos: > > Expectation is to iterate through a tuple like this: > > > > for dirpath, dirnames, filenames in os.walk(path): > > ... Thank you Michael, yes i ahve understood that myself yesterday after one of the guys here have shown the output of os.walk(path) in IDLE. So, yes in fact i was in need for the 3rd item in the tuple, so to get hold on to the files. Thank you for supporting me, some ppl here think i'am a troll and don't try thinks or ignore evrything but 'some_ppl's_opinion != True' I'am just sometimes persistant on having thing my way that why i was calling my self Ferrous cRanus, which i changes it since it was annoying -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating an object that can track when its attributes are modified
On Wednesday, 6 March 2013 16:22:56 UTC, Chris Angelico wrote: > > Effectively, you would need to have a > subclass of list/dict/tuple/whatever that can respond to the change. This is certainly something I'd be interested in having, but I guess that would be fragile since the user would have the burden of having to remember to use those types. > What's the goal of this class? Can you achieve the same thing by > using, perhaps, a before-and-after snapshot of a JSON-encoded form of > the object? > I need to be able to perform complex operations on the object that may modify several properties, and then gather the properties at the end as an efficient way to see what has changed and to store those changes. Any comparison of before-and-after snapshots could work in theory, but in practice it could be expensive to produce the snapshots on larger objects and probably expensive to calculate the differences that way too. Performance is important so I would probably just go for an explicit function call to mark an attribute as having been modified rather than trying to do a diff like that. (It wouldn't work for rollbacks, but I can accept that.) -- Ben Sizer -- http://mail.python.org/mailman/listinfo/python-list
Detecting the end of the program using bdb
Hello all, I am attempting to write a remote debugger using bdb. One problem I am running into is that if I call self.set_next when on the last line of my program, I see this line printed: Exception AttributeError: "'NoneType' object has no attribute 'path'" in ignored But I don't see any way of detecting that the program I am debugging has now completed. More generally, I am not sure how to use bdb (or some alternative way) to detect that the program has finished executing. Any help would be appreciated. Alexandre Zani -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating an object that can track when its attributes are modified
Ben Sizer writes: > I also believe that this won't catch modification to existing > attributes as opposed to assignments: eg. if one of the attributes is > a list and I append to it, this system won't notice. Is that something > I can rectify easily? It's really up to how far you wanna go: a similar use case is implemented by SQLAlchemy__, which "instrument" builtin collection classes to achieve the goal. But I'd not call that "easy" though :-) ciao, lele. __ http://www.sqlalchemy.org/trac/browser/lib/sqlalchemy/orm/collections.py -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929. -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating an object that can track when its attributes are modified
On Thu, Mar 7, 2013 at 3:56 AM, Ben Sizer wrote: > On Wednesday, 6 March 2013 16:22:56 UTC, Chris Angelico wrote: >> >> Effectively, you would need to have a >> subclass of list/dict/tuple/whatever that can respond to the change. > > This is certainly something I'd be interested in having, but I guess that > would be fragile since the user would have the burden of having to remember > to use those types. Since you're already overriding setattr, you could simply force all non-string sequences to your special subclass of list. That reduces that burden, though it'd break if there are any other references to the object. >> What's the goal of this class? Can you achieve the same thing by >> using, perhaps, a before-and-after snapshot of a JSON-encoded form of >> the object? >> > > I need to be able to perform complex operations on the object that may modify > several properties, and then gather the properties at the end as an efficient > way to see what has changed and to store those changes. Any comparison of > before-and-after snapshots could work in theory, but in practice it could be > expensive to produce the snapshots on larger objects and probably expensive > to calculate the differences that way too. Performance is important so I > would probably just go for an explicit function call to mark an attribute as > having been modified rather than trying to do a diff like that. (It wouldn't > work for rollbacks, but I can accept that.) Hmm. Interesting. The perfect solution probably is too messy, yeah. But if you have your subclassing done, you could possibly snapshot-on-write, which would allow the rollback. Not sure if it'd help though. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Getting started with Python: The ultimate guide with Tips, Tools and Resources
Getting started with Python: Tips, Tools and Resources http://lurnq.com/lesson/getting-started-with-python-tips-tools-and-resources/ This is a lesson I published on LurnQ which acts like a beginners guide. I have included various Books, MOOCs, Video Tutorials, Interactive tutorials, exercises which can get you started with Python. I have included my review along with some resources, which I have used while teaching Python myself. Any form of feedback is most welcome. Also, if I missed out on any good learning resource, do leave it as a comment. I will update the list. Thanks -- http://mail.python.org/mailman/listinfo/python-list
draw a line if the color of points of beginning and end are différent from white
how can i draw a line if the point of the begining and the end if those points are différent from the white in other exepretion how can i get the color of two points of the begining and the end? please help me -- http://mail.python.org/mailman/listinfo/python-list
Any logger created before calling logging.config.dictCOnfig is not configured
It seems like that any logger I create BEFORE calling
logging.config.dictConfig does not get configured.
Meanwhile, if I configure the logger like I always have, by just setting
handlers on root, everything works fine, including the loggers that were
created BEFORE I configure logging.
I make a lot of module-level log instances.
I wrote a simple script to show my problem. Run it like:
$ python scratch.py code
and then
$ python scratch.py dict
and see how the logging output is different.
### SCRIPT START
"""
import argparse
import logging
import logging.config
log1 = logging.getLogger('scratch')
def configure_logging_with_dictConfig():
d = {
'formatters': {
'consolefmt': {
'format': '%(asctime)s %(levelname)-10s %(process)-6d
%(name)-24s %(lineno)-4d %(message)s'}},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'consolefmt',
'level': 'DEBUG'}},
'root': {
'handlers': ['console'],
'level': 'DEBUG'},
'version': 1}
logging.config.dictConfig(d)
def configure_logging_with_code():
# Set the logger to DEBUG.
logging.root.setLevel(logging.DEBUG)
# Now write a custom formatter, so that we get all those different
# things.
f = logging.Formatter(
'%(asctime)s '
'%(levelname)-10s '
'%(process)-6d '
'%(filename)-24s '
'%(lineno)-4d '
'%(message)s '
)
# Set up a stream handler for DEBUG stuff (and greater).
sh = logging.StreamHandler()
sh.setLevel(logging.DEBUG)
sh.setFormatter(f)
logging.root.addHandler(sh)
def set_up_arguments():
ap = argparse.ArgumentParser()
ap.add_argument('how_to_configure', choices=['code', 'dict'])
return ap.parse_args()
if __name__ == '__main__':
args = set_up_arguments()
if args.how_to_configure == 'code':
configure_logging_with_code()
elif args.how_to_configure == 'dict':
configure_logging_with_dictConfig()
log1.debug('debug from log1')
# log2 is created AFTER I configure logging.
log2 = logging.getLogger('log2')
log2.debug('debug from log2')
# Try to figure out what is the difference! Nothing jumps out at me.
print "log.root.level: {0}".format(log1.root.level)
print "log.root.handlers: {0}".format(log1.root.handlers)
print "log1.parent.level: {0}".format(log1.parent.level)
print "log1.parent.handlers: {0}".format(log1.parent.handlers)
print "log1.level: {0}".format(log1.level)
print "log1.handlers: {0}".format(log1.handlers)
print "log1.propagate: {0}".format(log1.propagate)
print "log1.getEffectiveLevel(): {0}".format(log1.getEffectiveLevel())
### SCRIPT END
--
W. Matthew Wilson
m...@tplus1.com
http://tplus1.com
--
http://mail.python.org/mailman/listinfo/python-list
Why is Ruby on Rails more popular than Django?
I'm currently in the process of learning Ruby on Rails. I'm going through the Rails for Zombies tutorial, and I'm seeing the power of Rails. I still need to get a Ruby on Rails site up and running for the world to see. (My first serious RoR site will profile mutual funds from a value investor's point of view.) I have an existing web site and project called Doppler Value Investing (dopplervalueinvesting.com) that uses Drupal to display the web pages and Python web-scraping scripts to create *.csv and *.html files showing information on individual stocks. My site has a tacked-on feel to it, and I definitely want to change the setup. At a future time, I will rebuild my Doppler Value Investing web site in either Ruby on Rails or Django. The Ruby on Rails route will require rewriting my Python script in Ruby. The Django route will require learning Django. (I'm not sure which one will be easier.) My questions: 1. Why is Ruby on Rails much more popular than Django? 2. Why is there a much stronger demand for Ruby on Rails developers than Django/Python developers? 3. If Doppler Value Investing were your project instead of mine, would you recommend the Ruby on Rails route or the Django route? -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is Ruby on Rails more popular than Django?
> My questions: > 1. Why is Ruby on Rails much more popular than Django? AFAIK Rails got a slightly longer head start than Django. And it has been said that RoR's first killer app was a screencast. A little marketing can go a long way. Since then Django has caught up a bit with RoR in terms of maturity and adoption (I think this is in part because of RoR's adoption slowing due to it not being the NKOTB anymore (not to mention a few security embarrassments)) . > 2. Why is there a much stronger demand for Ruby on Rails developers than > Django/Python developers? I'm not sure how big the difference is, but it's probably related to its early(er) adoption. Same reason that there is a stronger demand for PHP coders. PHP hit it big first, so there is a lot more PHP code to maintain. > 3. If Doppler Value Investing were your project instead of mine, would > you recommend the Ruby on Rails route or the Django route? If you already know/work with Python than I would go the Django route. RoR and Django are not that much different nowadays as far as methodologies. The main differences I think between RoR and Django are that one is Ruby-based and one is Python-based. Other than that, if you can get used to one you can get used to the other. -- http://mail.python.org/mailman/listinfo/python-list
Re: Controlling number of zeros of exponent in scientific notation
One possibility is to form the string as usual, split on the "e", format each part separately, then rejoin with an "e". On Tuesday, March 5, 2013 12:09:10 PM UTC-8, fa...@squashclub.org wrote: > Instead of: > > > > 1.8e-04 > > > > I need: > > > > 1.8e-004 > > > > So two zeros before the 4, instead of the default 1. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is Ruby on Rails more popular than Django?
* mar...@python.net [130306 09:31]: > > > > > My questions: > > 1. Why is Ruby on Rails much more popular than Django? > If you already know/work with Python than I would go the Django route. > RoR and Django are not that much different nowadays as far as > methodologies. The main differences I think between RoR and Django are > that one is Ruby-based and one is Python-based. Other than that, if you > can get used to one you can get used to the other. I had problems getting django to work on my hostmonster account which is shared hosting and supports fast_cgi but not wsgi. I put that effort on hold for now, as it was just R&D for me, but I would welcome you to take a look at this link where I opened a ticket. https://code.djangoproject.com/ticket/19970 From what I inferred there and from the django ML, the django "community" is indifferent to fastcgi and the shared hosting environment. As someone is new to shared hosting environments (I would mostly on dedicated servers) I get the impression that django is cutting itself out of some (if not a lot) of the market. I don't know about RoR tho -- Tim tim at tee jay forty nine dot com or akwebsoft dot com http://www.akwebsoft.com -- http://mail.python.org/mailman/listinfo/python-list
Re: draw a line if the color of points of beginning and end are différent from white
On 06/03/2013 17:46, olsr.ka...@gmail.com wrote: how can i draw a line if the point of the begining and the end if those points are différent from the white in other exepretion how can i get the color of two points of the begining and the end? please help me Please tell us what package, Python version and OS, plus give us a code sample, what you expect to see and what actually happens. -- Cheers. Mark Lawrence -- http://mail.python.org/mailman/listinfo/python-list
Introducing Islam to Non--Muslims
Introducing Islam to Non--Muslims Shaikh Yusuf Estes Explains the basics of Islam including the five pillars of Islam to Non-Muslims. http://www.youtube.com/v/gBlCnpUkobE?rel=0 thank you -- http://mail.python.org/mailman/listinfo/python-list
Re: draw a line if the color of points of beginning and end are différent from white
On 03/06/2013 06:46 PM, olsr.ka...@gmail.com wrote: how can i draw a line if the point of the begining and the end if those points are différent from the white in other exepretion how can i get the color of two points of the begining and the end? please help me This should get you going. If it doesn't work it will still direct you to the relevant chapters in the tutorial. Frederic def draw_line (image): # image is a PIL Image ( ) # Define your colors WHITE = ~0 # Probably white for all modes. LINE_COLOR = 0 # define # Find end points points = [] pixels = image.load () # Fast pixel access for y in range (image.size [1]): for x in range (image.size [0]): if pixels [x, y] != WHITE points.append ((x, y)) # Join end points draw = ImageDraw.Draw (image) draw.line (points, fill = LINE_COLOR) -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating an object that can track when its attributes are modified
Ben Sizer於 2013年3月7日星期四UTC+8上午12時56分09秒寫道： > On Wednesday, 6 March 2013 16:22:56 UTC, Chris Angelico wrote: > > > > > > Effectively, you would need to have a > > > subclass of list/dict/tuple/whatever that can respond to the change. > > > > This is certainly something I'd be interested in having, but I guess that > would be fragile since the user would have the burden of having to remember > to use those types. > > > > > What's the goal of this class? Can you achieve the same thing by > > > using, perhaps, a before-and-after snapshot of a JSON-encoded form of > > > the object? > > > > > > > I need to be able to perform complex operations on the object that may modify > several properties, and then gather the properties at the end as an efficient > way to see what has changed and to store those changes. Any comparison of > before-and-after snapshots could work in theory, but in practice it could be > expensive to produce the snapshots on larger objects and probably expensive > to calculate the differences that way too. Performance is important so I > would probably just go for an explicit function call to mark an attribute as > having been modified rather than trying to do a diff like that. (It wouldn't > work for rollbacks, but I can accept that.) > > > > -- > > Ben Sizer Please hook a stack implemented as a list in python to every property of the object that you want to track down. -- http://mail.python.org/mailman/listinfo/python-list
debugging with idle
Hi, I am trying to debug a script using the idle debug control. I want to see the current source line which is executed, so I select the "Source" checkbox. If I step through the program, the editor window with the source is popping up, but the current source line is not marked. Only If I activate the source window by clicking on the window top, the marker appears. This I have to repeat for every step: click on "step" button, click on source window, click on "step" button, click on source window ... very boring. The problem occurs with python 3.3 on Windows XP SP3 as well as on Windows 7 X64, SP1. Running python 3.2 on Ubuntu 12.4 LTE the problem does not come up. Here the currently executed line stays marked regardless whether the source window is active or not. Any help is welcome. Regards Dirk -- http://mail.python.org/mailman/listinfo/python-list
Re: Inserting-embedding some html data at the end of a .py file
Τη Τετάρτη, 6 Μαρτίου 2013 2:06:33 π.μ. UTC+2, ο χρήστης Michael Ross έγραψε: > check_output is available as of Python 2.7 > I guess you are still on version 2.6 ? I can access each of these from my jailed shell user account without issue, and especially i try /usr/bin/python3 ni...@superhost.gr [~]# /usr/bin/python -V Python 2.6.6 ni...@superhost.gr [~]# /opt/python3/bin/python3 -V Python 3.2.3 ni...@superhost.gr [~]# /usr/bin/python3 -V Python 3.2.3 Problem is that when i change my shebang constructor to #!/sur/bin/python3 my '/cgi-bin/metrites.py' python cgi scripts produces an internal server error while with #!/sur/bin/python the script works. Any ideas as to why? I can post ocde if you want to. -- http://mail.python.org/mailman/listinfo/python-list
Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
On Wed, 06 Mar 2013 10:11:12 +, Wong Wah Meng-R32813 wrote: > Hello there, > > I am using python 2.7.1 built on HP-11.23 a Itanium 64 bit box. > > I discovered following behavior whereby the python process doesn't seem > to release memory utilized even after a variable is set to None, and > "deleted". I use glance tool to monitor the memory utilized by this > process. Obviously after the for loop is executed, the memory used by > this process has hiked to a few MB. However, after "del" is executed to > both I and str variables, the memory of that process still stays at > where it was. > > Any idea why? Python does not guarantee to return memory to the operating system. Whether it does or not depends on the OS, but as a general rule, you should expect that it will not. for i in range(10L): > ... str=str+"%s"%(i,) You should never build large strings in that way. It risks being horribly, horribly slow on some combinations of OS, Python implementation and version. Instead, you should do this: items = ["%s" % i for i in range(10)] s = ''.join(items) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Insert comma in number?
I have a python program that accepts input and calculates the factorial of that number, and i want to know if i can make it so commas get inserted in the number. For example: instead of 1000 it would say 1,000 -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On 3/6/2013 2:48 PM, rh wrote: I've tried twice to register with the bug tracker -- including just before sending this post. Both times I got something like this: Subject: Failed issue tracker submission From: Python tracker Date: Wed, 06 Mar 2013 00:56:44 + An unexpected error occurred during the processing of your message. The tracker administrator is being notified. So, it's not all that easy to report bugs for me, anyway. I'd given up, prior, but reading this thread I thought I'd try one more time. I had wanted to report doc bugs, too, as I used to do copy and line editing. I seem to be able to find typos and such rather easily, but reporting said bugs -- not so easy. From http://docs.python.org/3/bugs.html "Documentation bugs If you find a bug in this documentation or would like to propose an improvement, please send an e-mail to d...@python.org describing the bug and where you found it. If you have a suggestion how to fix it, include that as well. d...@python.org is a mailing list run by volunteers; your request will be noticed, even if it takes a while to be processed." For anything more than trivial changes (typos, grammar errors), a tracker issue is better. But the above is better than nothing. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Insert comma in number?
On 03/06/2013 03:39 PM, eli m wrote: I have a python program that accepts input and calculates the factorial of that number, and i want to know if i can make it so commas get inserted in the number. For example: instead of 1000 it would say 1,000 pip install humanize import humanize my_integer = 12345678 commafied_integer = humanize.intcomma(my_integer) print commafied_integer output: 12,345,678 -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is Ruby on Rails more popular than Django?
On Wed, 06 Mar 2013 10:03:14 -0800, Jason Hsu wrote: > My questions: > 1. Why is Ruby on Rails much more popular than Django? 2. Why is there > a much stronger demand for Ruby on Rails developers than Django/Python > developers? Fashion. Demand for technology is usually driven more by copying what everyone else does than by merit. Consider: Fred is a busy manager who has to start a new website and is dissatisfied with the technology he's previously been using. Does he have time to learn Ruby on Rails, Django, CherryPy, Drupal, and thirty other web technologies, to systematically and objectively decide on the best language for the website? Of course not. Even evaluating *two* technologies is probably beyond his time or budget constraints. So he does a search on the Internet, or reads trade magazines, or asks his peers, to find out what everyone else is doing, then copies them. "Oh, they're using Ruby on Rails, it must be good." So now he decides to use Ruby on Rails, advertises for RoR developers, and the cycle continues. But is RoR actually better for his specific situation? Doubtful. Presumably RoR is better for *some* specific jobs. At some point, early in RoR's history, it must have been a *good* solution. But unlikely to be the *best* solution, just better than whatever people were using before. And so RoR will be the easy choice, not the best choice, until such time as RoR is no longer satisfying developers. And then there will be a sudden, and random, phase-change to some other tool, which will become the next easy choice. > 3. If Doppler Value Investing were your project instead of > mine, would you recommend the Ruby on Rails route or the Django route? Neither. I'd be rather tempted to try doing it in CherryPy. But then, what do I know, I'm just as much a follow of fashion as the next guy. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Insert comma in number?
On Wed, Mar 6, 2013 at 3:39 PM, eli m wrote: > I have a python program that accepts input and calculates the factorial of > that number, and i want to know if i can make it so commas get inserted in > the number. > For example: instead of 1000 it would say 1,000 Use the "," (i.e. comma) format() specifier directive. See http://docs.python.org/2/library/string.html#format-specification-mini-language See also: http://www.python.org/dev/peps/pep-0378/ Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: debugging with idle
On 3/6/2013 5:30 PM, Dirk Zabel wrote: Hi, I am trying to debug a script using the idle debug control. I want to see the current source line which is executed, so I select the "Source" checkbox. If I step through the program, the editor window with the source is popping up, but the current source line is not marked. Only If I activate the source window by clicking on the window top, the marker appears. This I have to repeat for every step: click on "step" button, click on source window, click on "step" button, click on source window ... very boring. The problem occurs with python 3.3 on Windows XP SP3 as well as on Windows 7 X64, SP1. I use IDLE and work on IDLE issues, but have never used the debug window before. It looks useful -- and its operation should be documented. I believe I duplicated your report with 3.3 on win 7. Running python 3.2 on Ubuntu 12.4 LTE the problem does not come up. Here the currently executed line stays marked regardless whether the source window is active or not. You changed both python version and os. Which change made the difference? I believe I duplicated your problem also with 3.2 on win 7, so perhaps it is the OS. If you possibly can, load 3.3 on Ubuntu and see if you still get the correct behavior. Then open an issue on the tracker. bugs.python.org. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Inserting-embedding some html data at the end of a .py file
On Thu, 07 Mar 2013 00:18:44 +0100, Νίκος Γκρ33κ
wrote:
Τη Τετάρτη, 6 Μαρτίου 2013 2:06:33 π.μ. UTC+2, ο χρήστης Michael Ross
έγραψε:
check_output is available as of Python 2.7
I guess you are still on version 2.6 ?
I can access each of these from my jailed shell user account without
issue, and especially i try /usr/bin/python3
ni...@superhost.gr [~]# /usr/bin/python -V
Python 2.6.6
ni...@superhost.gr [~]# /opt/python3/bin/python3 -V
Python 3.2.3
ni...@superhost.gr [~]# /usr/bin/python3 -V
Python 3.2.3
Problem is that when i change my shebang constructor to
#!/sur/bin/python3 my '/cgi-bin/metrites.py' python cgi scripts produces
an internal server error while with #!/sur/bin/python the script works.
Any ideas as to why? I can post ocde if you want to.
You need to post the Traceback.
Either run /usr/bin/python3 /cgi-bin/metrites.py on the shell
or better look in your webserver error log.
Guess:
In Python 3 "print" is a function.
So
print "something"
will not work. You need to
print("something")
--
http://mail.python.org/mailman/listinfo/python-list
Re: Creating an object that can track when its attributes are modified
On Thursday, 7 March 2013 00:07:02 UTC, Steven D'Aprano wrote: > On Wed, 06 Mar 2013 08:56:09 -0800, Ben Sizer wrote: > > > I need to be able to perform complex operations on the object that may > > modify several properties, and then gather the properties at the end as > > an efficient way to see what has changed and to store those changes. Any > > comparison of before-and-after snapshots could work in theory, but in > > practice it could be expensive to produce the snapshots on larger > > objects and probably expensive to calculate the differences that way > > too. Performance is important so I would probably just go for an > > explicit function call to mark an attribute as having been modified > > rather than trying to do a diff like that. (It wouldn't work for > > rollbacks, but I can accept that.) > > Premature optimization. > > Unless you have been eating and breathing Python code for 15+ years, your > intuition of what is expensive and what isn't will probably be *way* off. > I've been using Python for ~15 years, and I wouldn't want to try to guess > what the most efficient way to do this will be. I admit, I've only been using Python for 10 years, but I've learned a lot about optimisation and what needs optimising from my time as a game developer. This code needs to be fairly high-performing due to the role it plays in my server and the frequency with which the behaviour gets called. > Actually I lie. I would guess that the simple, most obvious way is > faster: don't worry about storing what changed, just store *everything*. > But I could be wrong. The use case I have is not one where that is suitable. It's not the snapshots that are important, but the changes between them. > Fortunately, Python development is rapid enough that you can afford to > develop this object the straightforward way, profile your application to > see where the bottlenecks are, and if it turns out that the simple > approach is too expensive, then try something more complicated. I don't see a more straightforward solution to the problem I have than the one I have posted. I said that a system that took snapshots of the whole object and attempted to diff them would probably perform worse, but it would probably be more complex too, given the traversal and copying requirements. -- Ben Sizer -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Wednesday, March 6, 2013 5:50:35 PM UTC-6, Terry Reedy wrote: > If you find a bug in this documentation or would like to propose an > improvement, please send an e-mail to d...@python.org describing the bug > and where you found it. If you have a suggestion how to fix it, include > that as well. That's great Terry, but how will the next person find the link? I went to Python.org and i did not see it on the home page, nor the doc page... and i've been Python-ing for years now! If i can't find the link, how will a noob find it? How much longer are we going to "treat the symptoms" before we realize that we're dealing with a disease that can be cured? -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is Ruby on Rails more popular than Django?
On Mar 7, 9:58 am, Steven D'Aprano wrote: > Neither. I'd be rather tempted to try doing it in CherryPy. But then, > what do I know, I'm just as much a follow of fashion as the next guy. All of the cool kids are using Pyramid these days. -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Mar 7, 10:47 am, Rick Johnson wrote: > That's great Terry, but how will the next person find the link? Why do you have such a low opinion of others that you think they're unable to look up "Reporting Bugs" in the _documentation_? -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
I actually just tried that, and the results weren't very good. Using the doc's search feature, the "Reporting Bugs" (and the "About these documents") page was significantly down the page (about 2/3 of the way) - not the most obvious result in the pile. All the other searches I could think of either didn't return either of those pages at all, or they were also 1/2-2/3 of the way down the page. The link is also on the main documentation page, but it is, again, near the bottom, making it hidden from many users. Both of them show up just above the bottom of the window with it maximized on my screen. The first of the issues may be difficult to fix, but would likely be fairly useful - that would generally be my first port of call. The second one is more minor as most people will scroll down to see whats farther down, if they go to the main page to find the links. Chris On Wed, Mar 6, 2013 at 5:06 PM, alex23 wrote: > On Mar 7, 10:47 am, Rick Johnson wrote: > > That's great Terry, but how will the next person find the link? > > Why do you have such a low opinion of others that you think they're > unable to look up "Reporting Bugs" in the _documentation_? > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: Insert comma in number?
On 3/6/2013 7:07 PM, Chris Rebert wrote:
On Wed, Mar 6, 2013 at 3:39 PM, eli m wrote:
I have a python program that accepts input and calculates the factorial of that
number, and i want to know if i can make it so commas get inserted in the
number.
For example: instead of 1000 it would say 1,000
Use the "," (i.e. comma) format() specifier directive. See
http://docs.python.org/2/library/string.html#format-specification-mini-language
See also: http://www.python.org/dev/peps/pep-0378/
>>> format(12345234434, ',d')
'12,345,234,434'
>>> '{:,d}'.format()
'333,333,333,333'
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Wednesday, March 6, 2013 7:06:56 PM UTC-6, alex23 wrote: > Why do you have such a low opinion of others that you think they're > unable to look up "Reporting Bugs" in the _documentation_? I don't have a low opinion of anybody here. However the fact that this community needs an entry level path for bug/grievance reports is *glaringly* obvious. It would greatly benefit this community by: 1. Removing excess chatter from the bug tracker. We need to keep the tracker folks focused on *real* bugs that can be patched. Not engaged in endless discussions on the semantics of "what is a bug" and what "IS NOT a bug". 2. Removing barriers to reporting bugs/grievances. Remember, if reporting issues is too difficult, people will just give up. Then, the next person who gets slammed with the same problem will go through the same trouble only to produce no results AGAIN. Rinse and repeat! 3. Give us insight as to what aspects of the language/docs are troubling for folks. We need to know where the bottle necks are when learning the language, and since we are experienced, we lack the noob insight to even see the problems. I'll bet $100 you hated writing self as the first argument to each method. But now you've become so accustomed that you could so it in your sleep. That does not validate the asininity of doing such a thing! How can we fix entry-level problems if we DON'T KNOW WHAT THE PROBLEMS ARE! 4. Create a *real* sense of community. By creating a place for people to voice complaints, we are letting them know that "Hey, you opinion is important to us, even if you are a total NOOB!". Even if their "pet problem" is never solved (maybe because it was a misunderstanding all along), they are more likely to get involved more deeply in Python community later on. Heck, maybe they will work their way up to py-dev and rub shoulders with GvR one day, "the skys the limit"! I have already offered my assistance managing such a list. But i cannot start such a list without wide community support; I cannot start the list without GvR publicly supporting it; I cannot start the list without a prominent link on python.org; because if i do start a list without all these things, i will be wasting my time. -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On 3/6/2013 7:47 PM, Rick Johnson wrote: On Wednesday, March 6, 2013 5:50:35 PM UTC-6, Terry Reedy wrote: If you find a bug in this documentation or would like to propose an improvement, please send an e-mail to d...@python.org describing the bug and where you found it. If you have a suggestion how to fix it, include that as well. That's great Terry, but how will the next person find the link? The same way I did. Go to http://docs.python.org/3/ (or /2/ and click on Reporting bugs, which takes one to the above and more. > I went to Python.org and i did not see it on the home page, nor the doc page... Which doc page? > How much longer are we going to "treat the symptoms" We would VERY MUCH like a system to make it easier for readers to report doc bugs and developers to fix them. No one yet has come up with both a reasonable idea and workable implementation. Here is an idea I just came up with. *Someone* writes javascript that allows the following: Reader using the web version sees a mistake, selects some text with the mistake, right clicks, selects 'Suggest correction', gets a box or form with the selected text, page and location or context info, and a text entry box. Reader enters correction in text box and clicks OK. Message is emailed or posted to python.org. *Someone* writes a python script to process reports by generating a proposed patch for human review. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Thu, 07 Mar 2013 02:28:10 +0100, Chris Kaynor wrote: I actually just tried that, and the results weren't very good. Using the doc's search feature, the "Reporting Bugs" (and the "About these documents") page >was significantly down the page (about 2/3 of the way) - not the most obvious result in the pile. All the >other searches I could think of either didn't return either of those pages at all, or they were also 1/2-2/3 >of the way down the page. Using Google it took me about 3 seconds to find the page. "python report doc bug". Works with Bing or DuckDuckGo too. First hit on either engine. The doc's search is ... fine if I search for e. g. 'subprocess.Popen', but near-useless for general searches. The link is also on the main documentation page, but it is, again, near the bottom, making it hidden from >many users. Both of them show up just above the bottom of the window with it maximized on my screen. The first of the issues may be difficult to fix, Is it? You'd just have to have an additional search box labeld "search whole page with google/bing/whatever" (?) but would likely be fairly useful - that would generally be my first port of call. The second one is more >minor as most people will scroll down to see whats farther down, if they go to the main page to find the >links. Chris On Wed, Mar 6, 2013 at 5:06 PM, alex23 wrote: On Mar 7, 10:47 am, Rick Johnson wrote: That's great Terry, but how will the next person find the link? Why do you have such a low opinion of others that you think they're unable to look up "Reporting Bugs" in the _documentation_? -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is Ruby on Rails more popular than Django?
On Wed, Mar 6, 2013, at 02:16 PM, Tim Johnson wrote: > I had problems getting django to work on my hostmonster account > which is shared hosting and supports fast_cgi but not wsgi. I put > that effort on hold for now, as it was just R&D for me, but > I would welcome you to take a look at this link where I opened a > ticket. > https://code.djangoproject.com/ticket/19970 > From what I inferred there and from the django ML, the django > "community" is indifferent to fastcgi and the shared hosting > environment. As someone is new to shared hosting environments (I > would mostly on dedicated servers) I get the impression that > django is cutting itself out of some (if not a lot) of the market. > I don't know about RoR tho I haven't any experience with shared hosting, so can't help you there. I did do some work with lighttpd and fast_cgi and the Django docs worked fine for that. But you're right. wsgi is pretty much the standard for web services in Python, like DB API is to relational database access. Ruby has Rack. Python has WSGI. -- http://mail.python.org/mailman/listinfo/python-list
Re: why can not parse the web in almost same xpath expression?
python writes:
> import urllib
> import lxml.html
> down='http://v.163.com/special/visualizingdata/'
> file=urllib.urlopen(down).read()
> root=lxml.html.document_fromstring(file)
> urllist=root.xpath('//div[@class="down s-fc3 f-fl"]//a')
> for url in urllist:
> print url.get("href")
>
> i get the output ,
> http://mov.bn.netease.com/movieMP4/2012/12/A/7/S8H1TH9A7.mp4
> http://mov.bn.netease.com/movieMP4/2012/12/D/9/S8H1ULCD9.mp4
> http://mov.bn.netease.com/movieMP4/2012/12/4/P/S8H1UUH4P.mp4
> http://mov.bn.netease.com/movieMP4/2012/12/B/V/S8H1V8RBV.mp4
> http://mov.bn.netease.com/movieMP4/2012/12/6/E/S8H1VIF6E.mp4
> http://mov.bn.netease.com/movieMP4/2012/12/B/G/S8H1VQ2BG.mp4
>
> when i change
>
> xpath('//div[@class="down s-fc3 f-fl"]//a')
>
> into
>
> xpath('//div[@class="col f-cb"]//div[@class="down s-fc3 f-fl"]//a')
>
> that is to say ,
>
> urllist=root.xpath('//div[@class="col f-cb"]//div[@class="down s-fc3
> f-fl"]//a')
>
> why i can't get nothing?
There is only one in the document and that div
contains only a single but the latter does
not contain any . The URLs that you get in the first code are not
contained in a . They are contained in a , however.
So xpath('//div[@class="m-tdli"]//div[@class="down s-fc3 f-fl"]//a') works.
--
Piet van Oostrum
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]
--
http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Mar 7, 11:31 am, Rick Johnson wrote: > I don't have a low opinion of anybody here. However the fact that > this community needs an entry level path for bug/grievance reports > is *glaringly* obvious. Please explain how finding your vanity list would be easier than reading the Python doc's table of contents and clicking on the entry "Reporting Bugs". -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Wednesday, March 6, 2013 7:52:59 PM UTC-6, Terry Reedy wrote: > > How much longer are we going to "treat the symptoms" > > We would VERY MUCH like a system to make it easier for readers to report > doc bugs and developers to fix them. No one yet has come up with both a > reasonable idea and workable implementation. Here is an idea I just came > up with. > > *Someone* writes javascript that allows the following: Reader using the > web version sees a mistake, selects some text with the mistake, right > clicks, selects 'Suggest correction', gets a box or form with the > selected text, page and location or context info, and a text entry box. > Reader enters correction in text box and clicks OK. Message is emailed > or posted to python.org. YES, YES, YES! This is even better than "PyWarts" because the user will not need to log onto a forum and then compose a post. In effect, the forum will come to him! I love it! However, there is a dark-side on the opposite side of this mountain. Your idea solves "doc related" issues, but what about "language related" issues? I still love your idea, however, i am looking for a holistic approach to solving the issue. > *Someone* writes a python script to process reports by generating a > proposed patch for human review. Also, i would like to add. That all user submitted "reports" should be posted in a searchable database somewhere that is public; so we can keep track of which parts of the docs (or language) are creating the highest volume of complaints\bugs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Thu, Mar 7, 2013 at 12:31 PM, Rick Johnson wrote: > We need to know where the bottle necks are when learning the language, and > since we are experienced, we lack the noob insight to even see the problems. > I'll bet $100 you hated writing self as the first argument to each method. > But now you've become so accustomed that you could so it in your sleep. Gambling debts are due within twenty-four hours. Please remit that hundred dollars immediately; I had absolutely no problem with the explicit 'self' argument. Thanks, it's about time I earned some money arguing with you. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is Ruby on Rails more popular than Django?
* Albert Hopkins [130306 17:14]: > > > On Wed, Mar 6, 2013, at 02:16 PM, Tim Johnson wrote: > > > I had problems getting django to work on my hostmonster account > > which is shared hosting and supports fast_cgi but not wsgi. I put > > that effort on hold for now, as it was just R&D for me, but > > I would welcome you to take a look at this link where I opened a > > ticket. > > https://code.djangoproject.com/ticket/19970 > > From what I inferred there and from the django ML, the django > > "community" is indifferent to fastcgi and the shared hosting > > environment. As someone is new to shared hosting environments (I > > would mostly on dedicated servers) I get the impression that > > django is cutting itself out of some (if not a lot) of the market. > > I don't know about RoR tho > > I haven't any experience with shared hosting, so can't help you there. > I did do some work with lighttpd and fast_cgi and the Django docs worked > fine for that. But you're right. wsgi is pretty much the standard for > web services in Python, like DB API is to relational database access. > Ruby has Rack. Python has WSGI. I believe that indifference on the part of Python to fastcgi is a self-inflicted wound. I don't believe that there is any good excuse for such indifference, except for a sort of bureaucratic inertia. It's sad, when you consider how well python is designed and how crappily PHP is designed and how easy it is to set up and deploy drupal in the same environment. I speak from my own experience. respectfully : -- Tim tim at tee jay forty nine dot com or akwebsoft dot com http://www.akwebsoft.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is Ruby on Rails more popular than Django?
On Mar 6, 11:03 pm, Jason Hsu wrote: > I'm currently in the process of learning Ruby on Rails. I'm going through > the Rails for Zombies tutorial, and I'm seeing the power of Rails. > > I still need to get a Ruby on Rails site up and running for the world to see. > (My first serious RoR site will profile mutual funds from a value investor's > point of view.) > > I have an existing web site and project called Doppler Value Investing > (dopplervalueinvesting.com) that uses Drupal to display the web pages and > Python web-scraping scripts to create *.csv and *.html files showing > information on individual stocks. My site has a tacked-on feel to it, and I > definitely want to change the setup. > > At a future time, I will rebuild my Doppler Value Investing web site in > either Ruby on Rails or Django. The Ruby on Rails route will require > rewriting my Python script in Ruby. The Django route will require learning > Django. (I'm not sure which one will be easier.) > > My questions: > 1. Why is Ruby on Rails much more popular than Django? "Where there is choice there is no freedom" http://www.jiddu-krishnamurti.net/en/1954/1954-03-03-jiddu-krishnamurti-8th-public-talk Python-for-web offered so much choice -- zope, django, turbogears, cherrypy, web.py etc etc -- that the newbie was completely drowned. With Ruby there is only one choice to make -- choose Ruby and rails follows. Anyone who's used emacs will know this as the bane of FLOSS software -- 100 ways of doing something and none perfect -- IOW too much spurious choice. GvR understood and rigorously implemented a dictum that Nicklaus Wirth formulated decades ago -- "The most important thing about language design is what to leave out." Therefore Python is a beautiful language. Unfortunately the same leadership did not carry over to web frameworks and so we have a mess. I guess the situation is being corrected with google putting its artillery behind django. -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Wednesday, March 6, 2013 8:28:42 PM UTC-6, alex23 wrote: > On Mar 7, 11:31 am, Rick Johnson wrote: > > I don't have a low opinion of anybody here. However the fact that > > this community needs an entry level path for bug/grievance reports > > is *glaringly* obvious. > > Please explain how finding your vanity list would be easier than > reading the Python doc's table of contents and clicking on the entry > "Reporting Bugs". Firstly: It's not a "Rick's Vanity List" unless my name is on it. I don't expect to be named the "BDFL of PyWarts". Heck, i don't even expect to be "named" at all. GvR DOES NOT need to mention my name. All i am asking is that he show some support for the general *idea* of "lowering the bar for bug/grievance reporting". Or at least start by admitting we have a problem. Secondly: The "report bugs" feature of the doc is more concerned with "doc related" bugs. I want a holistic approach that will invite ALL Python related issues (docs, language, community, modules, 3rd party modules, etc...) to follow a linear path. There may be better ways of achieving my goals (f.e. Terry proposed a great idea). My point is that we need to lower the bar and try to integrate a linear path that will allow all levels of Python programmers to participate. -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Mar 7, 12:57 pm, Rick Johnson wrote: > GvR DOES NOT need to mention my name. All i am asking is that he show > some support for the general *idea* of "lowering the bar for bug/grievance > reporting". Or at least start by admitting we have a problem. Your obsession with Guido is tiring. Open source is not a cult of personality. It's about doing what you can where you can when you can to make things better. Insisting that he "endorse" your ideas is ridiculous. > Secondly: The "report bugs" feature of the doc is more concerned with > "doc related" bugs. It would really help your arguments if you actually spent some time investigating the issues you're ranting against: http://docs.python.org/3/bugs.html Documentation bugs are a brief paragraph at the top of the page, the rest of which addresses bugs with the language & standard library. > I want a holistic approach that will invite ALL Python related issues (docs, > language, community, modules, 3rd party modules, etc...) to follow a linear > path. Third party modules will never be handled by the Python bug tracker, nor should they be lumped into the same "path"; they're the concern of their developers who shouldn't be bound by your desire for a One True Way. Community "bugs" should be addressed on the python list. > My point is that we You keep saying "we" when you mean other people apart from yourself. If you have ideas for improvement, _then implement them_. The crate.io guys didn't wait for community validation to address what they perceived were issues with PyPI, they rolled up their sleeves and did something about it. -- http://mail.python.org/mailman/listinfo/python-list
Interesting list() un-optimization
I stumbled upon an interesting bit of trivia concerning lists and list comprehensions today. We use mongoengine as a database model layer. A mongoengine query returns an iterable object called a QuerySet. The "obvious" way to create a list of the query results would be: my_objects = list(my_query_set) and, indeed, that works. But, then I found this code: my_objects = [obj for obj in my_query_set] which seemed a bit silly. I called over the guy who wrote it and asked him why he didn't just write it using list(). I was astounded when it turned out there's a good reason! Apparently, list() has an "optimization" where it calls len() on its argument to try and discover the number of items it's going to put into the list. Presumably, list() uses this information to pre-allocate the right amount of memory the first time, without any resizing. If len() fails, it falls back to just iterating and resizing as needed. Normally, this would be a win. The problem is, QuerySets have a __len__() method. Calling it is a lot faster than iterating over the whole query set and counting the items, but it does result in an additional database query, which is a lot slower than the list resizing! Writing the code as a list comprehension prevents list() from trying to optimize when it shouldn't! -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting list() un-optimization
On 03/06/2013 10:20 PM, Roy Smith wrote: I stumbled upon an interesting bit of trivia concerning lists and list comprehensions today. We use mongoengine as a database model layer. A mongoengine query returns an iterable object called a QuerySet. The "obvious" way to create a list of the query results would be: my_objects = list(my_query_set) and, indeed, that works. But, then I found this code: my_objects = [obj for obj in my_query_set] which seemed a bit silly. I called over the guy who wrote it and asked him why he didn't just write it using list(). I was astounded when it turned out there's a good reason! Apparently, list() has an "optimization" where it calls len() on its argument to try and discover the number of items it's going to put into the list. Presumably, list() uses this information to pre-allocate the right amount of memory the first time, without any resizing. If len() fails, it falls back to just iterating and resizing as needed. Normally, this would be a win. The problem is, QuerySets have a __len__() method. Calling it is a lot faster than iterating over the whole query set and counting the items, but it does result in an additional database query, which is a lot slower than the list resizing! Writing the code as a list comprehension prevents list() from trying to optimize when it shouldn't! That is very interesting. list() assumes the __len__() method would be very quick. Perhaps list() should take an optional second argument that specifies the initial length to allocate. That way code that either doesn't want __len__() to be used, or that already knows a reasonable number to use, can supply the value to preallocate. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Do you feel bad because of the Python docs?
On Wednesday, March 6, 2013 9:12:37 PM UTC-6, alex23 wrote: > Your obsession with Guido is tiring. And your false accusations that i am somehow "obsessed" with GvR have BEEN tiring for quite some time! I am neither passionate for or prejudice against the man. I simple ask that he live up to his job title. > Open source is not a cult of > personality. It's about doing what you can where you can when you can > to make things better. Insisting that he "endorse" your ideas is > ridiculous. Of course insisting that he validate me *personally* would be ridiculous. But i am NOT suggesting that he validate ME, i am suggesting that he do his job. And his job is to oversee the language evolution and maintain a sense of community. Specifically: "leading by example". Q: "Why even have a python-list if GvR and all the "heavies" at py-dev never participate in the conversations?" By NOT participating they are speaking louder than words. > Third party modules will never be handled by the Python bug tracker, > nor should they be lumped into the same "path"; they're the concern of > their developers I agree, that was an unfortunate typo. > If you have ideas for improvement, _then implement them_. The crate.io > guys didn't wait for community validation to address what they > perceived were issues with PyPI, they rolled up their sleeves and did > something about it. Great, and i commend the contribution. But is yet ANOTHER Python package list going to help anyone? How about 10 or 20 more Python package indexes? Community fragmentation and language forks due to core-dev stubbornness is unfortunate. I would much rather had them contribute to the existing package index instead of creating a new one. Congratulations Alex, your solution of pushing everyone away is working flawlessly -- just don't be surprised when you find yourself cold and alone. -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting list() un-optimization
On 2013-03-06 22:20, Roy Smith wrote: > I stumbled upon an interesting bit of trivia concerning lists and > list comprehensions today. I agree with Dave Angel that this is interesting. A little testing shows that this can be rewritten as my_objects = list(iter(my_query_set)) which seems to then skip the costly __len__ call. Performance geeks are welcome to time it against the list-comprehension version :-) -tkc class Foo(object): def __init__(self): self.items = range(10) def __iter__(self): return iter(self.items) def __len__(self): print "Calling costly __len__" return len(self.items) print "Ensuring we can iterate over it:" for x in Foo(): print x print "\nJust call list():" lst = list(Foo()) print lst print "\nCall list(iter())" lst = list(iter(Foo())) print lst -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is Ruby on Rails more popular than Django?
On Mar 6, 11:03 pm, Jason Hsu wrote: > I'm currently in the process of learning Ruby on Rails. I'm going through > the Rails for Zombies tutorial, and I'm seeing the power of Rails. > > I still need to get a Ruby on Rails site up and running for the world to see. > (My first serious RoR site will profile mutual funds from a value investor's > point of view.) > > I have an existing web site and project called Doppler Value Investing > (dopplervalueinvesting.com) that uses Drupal to display the web pages and > Python web-scraping scripts to create *.csv and *.html files showing > information on individual stocks. My site has a tacked-on feel to it, and I > definitely want to change the setup. > > At a future time, I will rebuild my Doppler Value Investing web site in > either Ruby on Rails or Django. The Ruby on Rails route will require > rewriting my Python script in Ruby. The Django route will require learning > Django. (I'm not sure which one will be easier.) It is a natural programmer instinct that a uni-language solution is felt cleaner than a multi-language one. This feeling is valid under the following assumptions: - You are starting from ground up - The investment in learning something new is not considered significant In your case, with a site already up (maybe with a tacked on feel) and learning django a significant effort compared to directly coding in RoR, you should look at polyglot solutions more carefully (eg not directly relevant ... something like http://www.igvita.com/2009/03/20/ruby-polyglot-talking-with-erlang/ ) IOW code your site in RoR and call out to your python scraper-scripts may be an option to consider. -- http://mail.python.org/mailman/listinfo/python-list
Re: listbox binding..what is the current selection?
Thanks. I have spent time with the docs, at least with the Python v3.3 and tkinter v8.5 (pdf). I wish they had more examples. My approach is to browse the docs, try a program, fail, read the docs, try again. When I can't figure it out, I post. I appreciate the help. On Tuesday, March 5, 2013 8:26:12 PM UTC-5, Rick Johnson wrote: > On Tuesday, March 5, 2013 6:54:45 PM UTC-6, Rex Macey wrote: > > > I have a listbox with two strings "Fixed" and "Random". > > > [...] Here's the beginning of the set_lengthtype code: > > > > > > def set_lengthtype(event=None): > > >s=lbLengthtype.get(tk.ACTIVE) > > >print(s) > > >. > > > > > > The print(s) statement is for debugging. If 'Random' was > > > selected and the user clicks 'Fixed', then Random will > > > print. I would like to know what the user just selected, > > > not what was selected before the user clicked. How can I > > > determine the current selection? Thanks. > > > > Use "listbox.nearest(event.y)" instead of "get(ACTIVE"). And maybe you should > spend some time reading the docs eh? > > > > http://infohost.nmt.edu/tcc/help/pubs/tkinter/web/index.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Python SUDS issue
Hi Guys,
Not aware what "import" here is and what it will do.
But going through some google search result page found that there is
something called doctor in suds.
so tried changing code and it did fix the issue.
suds.TypeNotFound: Type not found: '(GetAccountBalanceFaultResponse,
http://www.payback.net/lmsglobal/xsd/v1/types, )'
Here is what i changed.
from suds.client import Client
from suds.xsd.doctor import Import, ImportDoctor
from suds.sax.element import Element
wsdlurl = 'https://46.51.221.138/PBExternalServices/v1/soap?wsdl'
schemaurl = 'http://www.payback.net/lmsglobal/xsd/v1/types'
schemaimport = Import(schemaurl)
schemadoctor = ImportDoctor(schemaimport)
client = Client(url=wsdlurl,doctor=schemadoctor)
print client
Printing client now lists all the methods and types associated with the
service.
But again stuck as on executing a method
response = client.service.GetAccountBalance(authtype)
replies with the error.
ValueError: unknown url type: {endpoint address}
Totally confused as what is wrong going on here.
Appreciate your help.
Regards,
VGNU
On Tue, Mar 5, 2013 at 12:46 PM, dieter wrote:
> VGNU Linux writes:
>
> > ...
> > Here is my code:
> > from suds.client import Client
> > wsdlurl = 'https://46.51.221.138/PBExternalServices/v1/soap?wsdl'
> > client = Client(wsdlurl)
> > print client
> >
> > And following is the error that occurs on trying to print client.
> > Traceback (most recent call last):
> > ...
> > dereference
> > midx, deps = x.dependencies()
> > File "C:\Python25\Lib\site-packages\suds\xsd\sxbasic.py", line 469, in
> > dependencies
> > raise TypeNotFound(self.ref)
> > suds.TypeNotFound: Type not found: '(GetAccountBalanceFaultResponse,
> > http://www.payback.net/lmsglobal/xsd/v1/types, )'
>
> Looks like a bug in the "WSDL" description of the web service.
> It seems to reference a type "GetAccountBalanceFaultResponse"
> associated with the namespace "
> http://www.payback.net/lmsglobal/xsd/v1/types";,
> but "suds" cannot find the type.
>
> Maybe an "import" is missing in the "WSDL" description.
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: listbox binding..what is the current selection?
On Wednesday, March 6, 2013 10:38:22 PM UTC-6, Rex Macey wrote: > I have spent time with the docs, at least with the Python > v3.3 and tkinter v8.5 (pdf). Could you post links to the documents you are reading please? Actually when i said "read the docs" i did not mean the offical Python docs on Tkinter (because, IMHO, they are woefully inadequate). I actually meant "read some online tutorials". The link i gave you is really more of a reference than a tutorial. It's my go-to reference when i can't remember the behavior of "this" or "that" method, or the spelling of "this" or "that" event sequence. Here is another link that is much more of a tutorial. http://effbot.org/tkinterbook/tkinter-whats-tkinter.htm > I wish they had more examples. Yes me too! Examples speak much louder than mere words. Of course, ONLY if the examples are good examples. Maybe we could combine my ability to write good examples with your ability to judge the noob worthiness of such examples and contribute something positive to this community of ingrates. (please don't mis-interpret the theatrical verbiage, i am, in fact, quite serious) > My approach is to browse the docs, try a program, fail, > read the docs, try again. When I can't figure it out, I > post. That's a wise and considerate path. Considerate both for yourself and others. The path to enlightenment The temptation to run and ask somebody for an answer is *VERY*, well, tempting. The problem is, the more you ask and receive *easy* answers, the more you become dependent on your teacher and not yourself. Luckily for me (although i was unaware of the benefit at the time) i was oblivious to online help forms when i learned programming; forcing me to be self-reliant when learning. It was just me, some badly written docs, and a text editor. :-) After lots of trial and error and many a smashed keyboard, i emerged from under my rock with a deep knowledge that i don't believe i could have attained by having someone spoon feed me the answers. It seems that when you *DO* find yourself in a wet paper bag that you can't seem to mentally punch yourself out of, and you are forced to get outside input, the last thing you need is people "gift wrapping" answers for you. A wise teacher will give you a few clues and then aim you in the correct direction, leaving you to find the exact path to enlightenment. Learning "how to problem solve" is much more important skill than "learning how to solve a problem"; the old "teach a man to fish..." thing. Request for your personal experiences: Since you are new to Tkinter i wonder if you can give some insights into the specific troubles you are encountering. I have long since forgotten most of the troubles i experienced. This would be a good chance to document the issues with the documents. Learning GUI's cuts you twice! Learning GUI's for the first time is very difficult because before you can learn the API of the current GUI library, you must first learn what a "EditText" is, and what a "StaticText" is, and a "ProgressBar", and a "NoteBook", and abstract things like "InputEvents", and how to bind "inputEvents", and "GeometryManagment"; and blah, blah, blah. This first step is where most GUI tutorials fail. They try to present both the widgets AND the code to control them simultaneously. It's no wonder the student becomes bewildered! I believe the first step should involve NO code or even code examples. The first step should be mainly a visual experience combined with interactive delving into the intricacies of what "actions" each widget will expose; using only natural language and visuals to describe the actions. But even when GUI tuts follow this wise progression they fail to utilize the power of visualization, and instead focus on pages and pages of boring text. The second step is learning the actual methods of the widgets and how to link code to actions. Morals of My Experience: Looking back i believe the Tkinter learning path is flawed in to general ways. 1. The official docs are woefully inadequate. Which could be solved by writing better documentation, or simply, by linking to a few of the existing good outside documentation. The two links i posted offer the best of both worlds (reference and tutorial[1]) But then, there are all the issues of poor examples around the web. I found myself being thrown into fits of confusion by reading documentations or tutorials that seemed to conflict with one
Re: Why is Ruby on Rails more popular than Django?
On Wednesday, March 6, 2013 8:58:12 PM UTC-6, rusi wrote: > "Where there is choice there is no freedom" > [snip link] > > Python-for-web offered so much choice -- zope, django, turbogears, > cherrypy, web.py etc etc -- that the newbie was completely drowned. > With Ruby there is only one choice to make -- choose Ruby and rails > follows. Indeed! "Costco", a wholesale grocery chain, realized the same issue of consumers being drowned by multiplicity, and have been very successful by intelligently narrowing those choices for it's customer base. -- http://mail.python.org/mailman/listinfo/python-list
Unhelpful traceback
Here's a traceback that's not helping:
Traceback (most recent call last):
File "InfoCompaniesHouse.py", line 255, in
main()
File "InfoCompaniesHouse.py", line 251, in main
loader.dofile(infile) # load this file
File "InfoCompaniesHouse.py", line 213, in dofile
self.dofilezip(infilename) # do ZIP file
File "InfoCompaniesHouse.py", line 198, in dofilezip
self.dofilecsv(infile, infd)# as a CSV file
File "InfoCompaniesHouse.py", line 182, in dofilecsv
for fields in reader : # read entire
CSV file
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in
position 14: ordinal not in range(128)
This is wierd, becuase "for fields in reader" isn't directly
doing a decode. That's further down somewhere, and the backtrace
didn't tell me where.
The program is converting some .CSV files that come packaged in .ZIP
files. The files are big, so rather than expanding them, they're
read directly from the ZIP files and processed through the ZIP
and CSV modules.
Here's the code that's causing the error above:
decoder = codecs.getreader('utf-8')
with decoder(infdraw,errors="replace") as infd :
with codecs.open(outfilename, encoding='utf-8', mode='w') as
outfd :
headerline = infd.readline()
self.doheaderline(headerline)
reader = csv.reader(infd, delimiter=',', quotechar='"')
for fields in reader :
pass
Normally, the "pass" is a call to something that
uses the data, but for test purposes, I put a "pass" in there. It still
fails. With that "pass", nothing is ever written to the
output file, and no "encoding" should be taking place.
"infdraw" is a stream from the zip module, create like this:
with inzip.open(zipelt.filename,"r") as infd :
self.dofilecsv(infile, infd)
This works for data records that are pure ASCII, but as soon as some
non-ASCII character comes through, it fails.
Where is the error being generated? I'm not seeing any place
where there's a conversion to ASCII. Not even a print.
John Nagle
--
http://mail.python.org/mailman/listinfo/python-list
Re: Inserting-embedding some html data at the end of a .py file
Τη Πέμπτη, 7 Μαρτίου 2013 2:25:09 π.μ. UTC+2, ο χρήστης Michael Ross έγραψε:
> Either run /usr/bin/python3 /cgi-bin/metrites.py on the shell
> or better look in your webserver error log.
> Guess:
> In Python 3 "print" is a function.
> So
> print "something"
> will not work. You need to
>
> print("something")
Yes Michael i have already prinr with parenthesis except form the triple
quoting:
print ''' LOG όλων των HTML σελίδων
ΣελίδαΕπισκέψεις
'''
have turned to:
print(''' LOG όλων των
HTML σελίδων
ΣελίδαΕπισκέψεις
''')
but now iam receiving this error concering except:
ni...@superhost.gr [~/www/cgi-bin]# /usr/bin/python3 metrites.py
File "metrites.py", line 88
except MySQLdb.Error, e:
^
SyntaxError: invalid syntax
ni...@superhost.gr [~/www/cgi-bin]#
which used to work ok in v2.6.6
can you help?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Inserting-embedding some html data at the end of a .py file
The whole try stement is as follows to have the compete idea: try: cur.execute( '''SELECT url, hits FROM counters ORDER BY hits DESC''' ) data = cur.fetchall() for row in data: (url, hits) = row print( " %s " ) % (url, url) print( " %s " ) % (hits) except MySQLdb.Error, e: print( "Query Error: ", sys.exc_info()[1].excepinfo()[2] ) -- http://mail.python.org/mailman/listinfo/python-list
RE: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
Python does not guarantee to return memory to the operating system. Whether it does or not depends on the OS, but as a general rule, you should expect that it will not. for i in range(10L): > ... str=str+"%s"%(i,) You should never build large strings in that way. It risks being horribly, horribly slow on some combinations of OS, Python implementation and version. Instead, you should do this: items = ["%s" % i for i in range(10)] s = ''.join(items) [] The example is written for illustration purpose. Thanks for pointing out a better way of achieving the same result. Yes it seems so that the OS thinks the piece allocated to Python should not be taken back unless the process dies. :( -- http://mail.python.org/mailman/listinfo/python-list
Re: Unhelpful traceback
On 2013.03.07 00:33, John Nagle wrote: > This is wierd, becuase "for fields in reader" isn't directly > doing a decode. That's further down somewhere, and the backtrace > didn't tell me where. Looking at the csv module docs,the reader object iterates over the csvfile argument (which can be any iterator). I think that, in the case of a file object, it's not decoded until iteration. I've never used the csv module before though, so I could be wrong. -- CPython 3.3.0 | Windows NT 6.2.9200 / FreeBSD 9.1 -- http://mail.python.org/mailman/listinfo/python-list
Re: Unhelpful traceback
On Wed, Mar 6, 2013 at 10:33 PM, John Nagle wrote: > Here's a traceback that's not helping: > UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in > position 14: ordinal not in range(128) > The program is converting some .CSV files that come packaged in .ZIP > files. The files are big, so rather than expanding them, they're > read directly from the ZIP files and processed through the ZIP > and CSV modules. > This works for data records that are pure ASCII, but as soon as some > non-ASCII character comes through, it fails. I'd recommend using the `unicodecsv` package, which, unlike the std lib `csv` module, is properly Unicode-compatible: https://pypi.python.org/pypi/unicodecsv Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)
On Thu, Mar 7, 2013 at 5:33 PM, Wong Wah Meng-R32813 wrote: > [] The example is written for illustration purpose. Thanks for pointing out a > better way of achieving the same result. Yes it seems so that the OS thinks > the piece allocated to Python should not be taken back unless the process > dies. :( Don't be too bothered by that. That memory will be reused by Python for subsequent allocations. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Python SUDS issue
VGNU Linux wrote at 2013-3-7 10:07 +0530:
>Not aware what "import" here is and what it will do.
XML-schema has an "import" facility to modularize schema descriptions.
It is very similar to the "import" facilities you know from Python
(and a lot of other languages) -- and has very similar purpose.
> ...
>Here is what i changed.
>from suds.client import Client
>from suds.xsd.doctor import Import, ImportDoctor
>from suds.sax.element import Element
>
>wsdlurl = 'https://46.51.221.138/PBExternalServices/v1/soap?wsdl'
>schemaurl = 'http://www.payback.net/lmsglobal/xsd/v1/types'
>schemaimport = Import(schemaurl)
>schemadoctor = ImportDoctor(schemaimport)
>client = Client(url=wsdlurl,doctor=schemadoctor)
>print client
>
>Printing client now lists all the methods and types associated with the
>service.
>But again stuck as on executing a method
>response = client.service.GetAccountBalance(authtype)
>replies with the error.
>ValueError: unknown url type: {endpoint address}
>
>Totally confused as what is wrong going on here.
Again, this looks like a bug in the WSDL description of the
web service.
To understand the problem better, you should look at the
corresponding traceback (as always). It will tell you
where the "ValueError" comes from. At the corresponding
code location, you should learn whether "{endpoint address}"
is the literal value of the bad url or an indication given
by "suds" where it has found the bad url.
In the first case, search the WSDL for "endpoint address" -
and then get it fixed.
In the second case, the bad value likely comes from
the "location" attribute of a "soap:address" element below
"wsdl:service" (again in your WSDL). Again, get it fixed, then.
--
Dieter
--
http://mail.python.org/mailman/listinfo/python-list
Re: Interesting list() un-optimization
Roy Smith wrote: > I stumbled upon an interesting bit of trivia concerning lists and list > comprehensions today. > > We use mongoengine as a database model layer. A mongoengine query > returns an iterable object called a QuerySet. The "obvious" way to > create a list of the query results would be: > > my_objects = list(my_query_set) > > and, indeed, that works. But, then I found this code: > >my_objects = [obj for obj in my_query_set] > > which seemed a bit silly. I called over the guy who wrote it and asked > him why he didn't just write it using list(). I was astounded when it > turned out there's a good reason! > > Apparently, list() has an "optimization" where it calls len() on its > argument to try and discover the number of items it's going to put into > the list. Presumably, list() uses this information to pre-allocate the > right amount of memory the first time, without any resizing. If len() > fails, it falls back to just iterating and resizing as needed. > Normally, this would be a win. > > The problem is, QuerySets have a __len__() method. Calling it is a lot > faster than iterating over the whole query set and counting the items, > but it does result in an additional database query, which is a lot > slower than the list resizing! Writing the code as a list comprehension > prevents list() from trying to optimize when it shouldn't! Interesting discovery. Yet isn't this as much an issue with the mongoengine library as with list()? Queryset.count() can be called if the "length" of a resultset needs to be retrieved, so the __len__() methd seems redundant. And given that it's not unheard of to call list() on iterables, perhaps the library designers should either optimise the __len__() method, or document the performance implications of calling list on the queryset? Anyway, thanks for this thought-provoking post. Cheers, Kev -- http://mail.python.org/mailman/listinfo/python-list
Re: Insert comma in number?
eli m wrote:
> I have a python program that accepts input and calculates the factorial of
> that number, and i want to know if i can make it so commas get inserted in
> the number. For example: instead of 1000 it would say 1,000
Last not least there's the option to employ locale-aware formatting:
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "en_US.UTF-8")
'en_US.UTF-8'
>>> locale.format("%d", 12345, grouping=True)
'12,345'
In German usage of "." and "," is reversed, so:
>>> locale.setlocale(locale.LC_ALL, "de_DE.UTF-8")
'de_DE.UTF-8'
>>> locale.format("%d", 12345, grouping=True)
'12.345'
--
http://mail.python.org/mailman/listinfo/python-list
