Resetting Signal Mask
Hi, with mod_wsgi (apache2) a process created with os.system() has a modified signal mask, that SIGPWR gets ignored. A small wrapper (see bottom) resets the signal mask and uses execv to run the programm. Unfortunately python does not support sigprocmask. Is there any other way to reset the signal mask? I guess it could be done with ctypes (I never used it up to now). Why is sigprocmask not available in Python? Python exposes most of the other POSIX API. Thomas 1 #include 2 #include 3 #include 4 5 int main(int argc, char *argv[]) { 6 sigset_t newmask; 7 8 sigemptyset(&newmask); 9 sigprocmask(SIG_SETMASK, &newmask, NULL); 10 int ret=execv(argv[1], &argv[1]); 11 if (ret != 0) { perror(argv[0]); } 12 return ret; 13 } -- http://mail.python.org/mailman/listinfo/python-list
Re: Cursors in a Loop
On Jan 4, 5:11 am, Carsten Haese <[EMAIL PROTECTED]> wrote: > On Thu, 2008-01-03 at 17:25 -0800, t_rectenwald wrote: > > On Jan 3, 7:47 pm, t_rectenwald <[EMAIL PROTECTED]> wrote: > > > I have a python script that uses the cx_Oracle module. I have a list > > > of values that I iterate through via a for loop and then insert into > > > the database. This works okay, but I'm not sure whether I can use one > > > cursor for all inserts, and define it outside of the loop, or > > > instantiate and close the cursor within the loop itself. For example, > > > I have: > > > > for i in hostlist: > > > cursor = connection.cursor() > > > sql= "insert into as_siebel_hosts_temp values('%s')" % (i) > > > cursor.execute(sql) > > > cursor.close() > > > > And I've also tried: > > > > cursor = connection.cursor() > > > for i in hostlist: > > > sql= "insert into as_siebel_hosts_temp values('%s')" % (i) > > > cursor.execute(sql) > > > cursor.close() > > > > Both work fine, and execute in the same amount of time. I'm just > > > trying to understand what is the "correct" approach to use. > > Even better would be to use executemany: > > cursor = connection.cursor() > cursor.executemany("insert into as_siebel_hosts_temp values(?)", > [(i,) for i in hostlist] ) > cursor.close() > > Depending on whether cx_Oracle allows this, the list comprehension in > that example could be replaced by the generator expression > ((i,) for i in hostlist), but I don't know if cx_Oracle allows > executemany with an arbitrary iterable. You should bind all variables to save the pool. cursor = connection.cursor() cursor.executemany("""insert into as_siebel_hosts_temp values (:whole, :lot, :of, :bind, :variables) """ ,[(i,)[0] for i in hostlist] ) connection.commit() connection.close() -- http://mail.python.org/mailman/listinfo/python-list
Re: reassign to builtin possible !?
On Jan 3, 8:06 pm, "Chris Mellon" <[EMAIL PROTECTED]> wrote: > In Py3k this will be a syntax error, like assigning to None is now. > Possibly also in 2.6. thanks. I feed much better with that :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Adding a HTTP header to a SOAPpy request
On Jan 3, 5:43 pm, Matias Surdi <[EMAIL PROTECTED]> wrote: > Hi, > > Could anybody tell me which is the easier way to do a SOAP call to a web > service wich requires an http header to be present? > > I can't figure it out. > > Thanks a lot > > Some code I'm using: > > import SOAPpy > s = > SOAPpy.SOAPProxy("http://10.3.5.128:10560/SERVICES",namespace="http://ws.mysite.com";) > > s.some_method() > > Thanks a lot. Hallo, look at http://pywebsvcs.sourceforge.net. There is a mailing list about SOAP and Python concerning SOAPpy. Greetings Rainer -- http://mail.python.org/mailman/listinfo/python-list
Jython: Honourable mention on Charles Nutter's blog.
Hi all, The Jython team got an honourable mention on the Headius blog this morning. Apparently they have got Django working with the latest builds of Jython: http://headius.blogspot.com/2008/01/jythons-back-baby.html So congratulations! -- Ant. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyObject_CallObject code dump after calling 4 times
grbgooglefan wrote: > char* plevel = NULL; > if(NULL != (plevel = PyString_AsString(pResult))){ > ret = 0; > strncpy(szEvalResult,plevel,strlen(plevel)); strncpy doesn't check the size of the target buffer, so that's no different from just doing strcpy(szEvalResult, plevel). or in other words, it's still trivial to crash your program simply by returning too much data from the Python code. -- http://mail.python.org/mailman/listinfo/python-list
Re: dictionary/hash and '1' versus 1
Reedick, Andrew wrote: > As a Perl monkey in the process of learning Python, I just stepped on > the "'1' (string) is not the same as 1 (integer) in regards to keys for > dictionaries/hashes" landmine. This isn't a landmine; this is a _good_ thing. Python is strongly typed. > Is there a good way to ensure that > numbers represented as strings or ints do not get mixed up as keys? Convert them all to either strings or integers (whichever is more useful) before you add them to the dictionary, really. > It's fugly to wrap every key reference in str(), ex: > foo[str(some_func(i))]. Then wrap it in a function or a method of one of your classes. You only need to write it once. > It's tedious to add a has_key before every key > lookup. There's no need to do this, though you don't say why you're bothering to. Either use .setdefault, or just query and get the exception, or just insert the new key, value pair to override the contents. > Any good solutions or accepted practices to prevent the intermixing of > number strings and integers as hash keys? A hash wrapper class seems to > be the best bet so far. If you want to make sure something is always done in a particular situation, then solution is to have a function or method that does that, and then just call that function or method. That's true in any language -- any one that has functions, anyway. -- Erik Max Francis && [EMAIL PROTECTED] && http://www.alcyone.com/max/ San Jose, CA, USA && 37 18 N 121 57 W && AIM, Y!M erikmaxfrancis Laws are silent in time of war. -- Cicero -- http://mail.python.org/mailman/listinfo/python-list
Re: linecache and glob
jo3c wrote: > i have a 2000 files with header and data > i need to get the date information from the header > then insert it into my database > i am doing it in batch so i use glob.glob('/mydata/*/*/*.txt') > to get the date on line 4 in the txt file i use > linecache.getline('/mydata/myfile.txt/, 4) > > but if i use > linecache.getline('glob.glob('/mydata/*/*/*.txt', 4) won't work glob.glob returns a list of filenames, so you need to call getline once for each file in the list. but using linecache is absolutely the wrong tool for this; it's designed for *repeated* access to arbitrary lines in a file, so it keeps all the data in memory. that is, all the lines, for all 2000 files. if the files are small, and you want to keep the code short, it's easier to just grab the file's content and using indexing on the resulting list: for filename in glob.glob('/mydata/*/*/*.txt'): line = list(open(filename))[4-1] ... do something with line ... (note that line numbers usually start with 1, but Python's list indexing starts at 0). if the files might be large, use something like this instead: for filename in glob.glob('/mydata/*/*/*.txt'): f = open(filename) # skip first three lines f.readline(); f.readline(); f.readline() # grab the line we want line = f.readline() ... do something with line ... -- http://mail.python.org/mailman/listinfo/python-list
Re: C++ equivalent of comp.lang.python?
Russ P. wrote: >> make sense either. As an example, I was recently trying to get >> information about writing cross-platform code for dynamic linking, but >> I couldn't find anywhere appropriate to ask about it. > > Well, if the good folks at comp.lang.c++ can't even direct you to an > appropriate forum on C++, then I doubt the folks at comp.lang.python > can. I suggest you abandon C++ and try Python, Java, or Ada. note that for his specific example, we would of course direct him to the relevant portions of the CPython source code. -- http://mail.python.org/mailman/listinfo/python-list
NetSpeed
Hey can i check the average Downloading speed of available internet connection without downloading any dummy file? :o -- Sunil Ghai -- http://mail.python.org/mailman/listinfo/python-list
Re: Who's to blame?
Hallo, Mike. First of all, thanks to both you and Rob for your answers. I now see that the wxPython group would have been a better place to post to, all the more so given the tight connection between the wxPython and wxWidgets projects, of which at first I wasn't aware. On Jan 3, 8:19 pm, [EMAIL PROTECTED] wrote: [...] > I've never created a modal dialog like this. Instead, I follow the > wxPython in Action book examples (most of the time), which would do a > yes/no dialog like this: > > > > dlg = wx.MessageDialog(None, 'Some Message', 'A Message Box', > wx.YES_NO | wx.QUESTION) > retCode = dlg.ShowModal() > if retCode == wx.ID_YES: ># do something >print 'yes' > else: ># do something else >print 'no' > dlg.Destroy() > > Actually my example started out as something like if wx.MessageBox(message="Some message", caption="Some caption", style=wx.YES|wx.NO) == wx.YES: pass I had to change it because the actual message can become very long, so I assembled a dialog with a scrollable text field. Maybe I'm expecting to much of wxStdDialogButtonSizer, but I still feel that given that both your method and mine above work straight away, it should provide the same behaviour with Yes/No buttons as with OK/Cancel ones. Cheers, Nicola Musatti -- http://mail.python.org/mailman/listinfo/python-list
import zlib in 2.5 fails
import zlib works in Python 2.4 (debian etch AMD64 - default python version for that distro) I built python 2.5 from source; zlib is not importable. I am trying to compile MySQLdb. any clues about how to get zlib able to be imported in 2.5? -sg -- http://mail.python.org/mailman/listinfo/python-list
adding class functionality, nested scoping
Hi, regarding automatically adding functionality to a class (basically taken from the cookbook recipee) and Python's lexical nested scoping I have a question wrt this code: #- import types # minor variation on cookbook recipee def enhance_method(cls, methodname, replacement): 'replace a method with an enhancement' method = getattr(cls, methodname) def _f(*args, **kwargs): return replacement(method, *args, **kwargs) _f.__name__ = methodname setattr(cls, methodname, types.MethodType(_f, None, cls)) # loop over class dict and call enhance_method() function # for all methods to modify def enhance_all_methods(cls, replacement): for methodname in cls.__dict__: if not methodname.startswith("__"): method = getattr(cls, methodname) def _f(*args, **kwargs): return replacement(method, *args, **kwargs) _f.__name__ = methodname enhance_method(cls, methodname, replacement) # Does not work: all enhanced methods only call the last wrapped originial # method. It seems the name 'method' in the surrounding scope of the # def _(...) function definition only refers to the last loop value(?) def ERRONEOUS_enhance_all_methods(cls, replacement): for methodname in cls.__dict__: if not methodname.startswith("__"): method = getattr(cls, methodname) def _f(*args, **kwargs): return replacement(method, *args, **kwargs) _f.__name__ = methodname setattr(cls, methodname, types.MethodType(_f, None, cls)) class Foo(object): def foo(self, x): print "foo", x def bar(self, x): print "bar", x def logme(method, *args, **kwargs): print "-->", method.__name__, args, kwargs try: return method(*args, **kwargs) finally: print "<--" #enhance_all_methods(Foo, logme) ERRONEOUS_enhance_all_methods(Foo, logme) foo = Foo() foo.foo(2) foo.bar(2) #- ...give this output: >>> foo = Foo() >>> foo.foo(2) --> foo (<__main__.Foo object at 0x1b08f0>, 2) {} foo 2 <-- >>> foo.bar(2) --> foo (<__main__.Foo object at 0x1b08f0>, 2) {} foo 2 <-- >>> So, while using enhance_all_methods() to add functionality does work, ERRONEOUS_enhance_all_methods() does not. Why is this? Is the explanation I tried to give in the code comment on the right track: # Does not work: all enhanced methods only call the last wrapped originial # method. It seems the name 'method' in the surrounding scope of the # def _(...) function definition only refers to the last loop value(?) Thanks for any hint, Holger -- Pt! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger?did=10 -- http://mail.python.org/mailman/listinfo/python-list
Why python says "unexpected parameter 'mini.py'" for my code?
The following is my pure-python wxwidgets test. It runs and give a frame, but soon python comes up to say "unexpected parameter 'mini.py'" and I have to close it. I cannot find the reason. Can somebody give me a hint to let it work well? Thanks http://pyguiviactypes.googlepages.com/mini.py -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with global var
Bruno Ferreira wrote: > I wrote a very simple python program to generate a sorted list of > lines from a squid access log file. Now that your immediate problem is solved it's time to look at the heapq module. It solves the problem of finding the N largest items in a list much more efficiently. I think the following does the same as your code: import heapq def key(record): return int(record[4]) logfile = open("squid_access.log", "r") records = (line.split() for line in logfile) topsquid = heapq.nlargest(50, records, key=key) for record in topsquid: print record[4] Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: NetSpeed
-On [20080104 10:34], Sunil Ghai ([EMAIL PROTECTED]) wrote: >Hey can i check the average Downloading speed of available internet connection >without downloading any dummy file? :o For all I know you cannot check anything until you have established some data over a given period of time. So at the start it will be: N/A Only after data has been passing through the connection and it having been measured can you calculate an average. But perhaps I missed something. -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ Death is that state where one exists only in the memories of others... -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 disable proxy
Hi Dimitris, > I've been looking for a way to explicitly disable the use of proxies with > urllib2, no matter what the environment dictates. Unfortunately I can't find > a way [...] Would changing the environment work? Like this: >>> del os.environ['http_proxy'] >>> do_stuff_with_urllib2() -- Richie Hindle [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: import zlib in 2.5 fails
Modules/main.c:186: warning: function declaration isn't a prototype /home/name/Desktop/webdl/Python-2.5.1/Modules/_ctypes/libffi/src/x86/ ffi64.c:45: warning: function declaration isn't a prototype /home/name/Desktop/webdl/Python-2.5.1/Modules/_ctypes/libffi/src/x86/ ffi64.c:342: warning: function declaration isn't a prototype /usr/bin/ld: /usr/local/lib/libz.a(adler32.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /usr/local/lib/libz.a: could not read symbols: Bad value collect2: ld returned 1 exit status this error occurred at one point during a compilation of Python 2.5. It seems related to my inability to import zlib now. On Jan 4, 10:19 am, stuntgoat <[EMAIL PROTECTED]> wrote: > import zlib works in Python 2.4 (debian etch AMD64 - default python > version for that distro) > > I built python 2.5 from source; zlib is not importable. > > I am trying to compile MySQLdb. > > any clues about how to get zlib able to be imported in 2.5? > > -sg -- http://mail.python.org/mailman/listinfo/python-list
storing setup.py (for py2exe) in other directory than main script directory
Hello list, I have the following problem after upgrading to python 2.5 and py2exe 0.6.6 It seems that py2exe is using the location of the setup.py file as its search path, instead of the current working folder. (This used to be different with an older version of py2exe) Can I tell py2exe somehow to use the current working directory as search path instead of the location of the setup.py file ? Also: is this a feature or a bug ? The reason I am asking is because I'd like to keep the build scripts in a "win32specific" subfolder of my code folder, and then run the following command in the code folder: python win32specific\setup.py py2exe Alternative suggestions also welcome! Best regards, Stefaan. -- http://mail.python.org/mailman/listinfo/python-list
Re: choosing random dynamic port number
On 3 Gen, 23:21, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Emin.shopper Martinian.shopper wrote: > > Is there a good way to choose/assign random dynamic port numbers in python? > > > I had in mind something like the following, but if multiple programs are > > generating random port numbers, is there a way to check if a given port > > number is already taken? > > > def GenerateDynamicPortNumber(): > > "Generate a random dynamic port number and return it." > > # port numbers between 49152 to 65535 are dynamic port numbers > > return 49152 + random.randrange(15000) > > def GenerateDynamicPortNumber(): > return 0 > > (to get the actual number, use getsockname() on the socket after you've > called "bind" on it) > > By using 0 as port number value you let kernel choose a free unprivileged random port: >>> import socket >>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >>> s.bind(('', 0)) >>> s.getsockname() ('0.0.0.0', 3070) -- http://mail.python.org/mailman/listinfo/python-list
how to connect to a remote machine using python........
I am using pexpect with Cgi to ssh into remote machine,Whenever i run a script which does ssh using pexpect from the linux terminal it gives me the correct result, but when i call the script using commands module from the cgi program i am getting the following error.(0, '(256, \'Traceback (most recent call last):\\n File "/usr/local/https/suid/getdetails.py", line 85, in \\nmain()\\n File "/usr/local/https/suid/getdetails.py", line 66, in main\\nmac = RemoteCommand(command,passwd)\\n File "/usr/local/https/suid/getdetails.py", line 42, in RemoteCommand\\nchild = pexpect.spawn(command)\\n File "/usr/lib/python2.5/site-packages/pexpect.py", line 375, in __init__\\nself.__spawn()\\n File "/usr/lib/python2.5/site-packages/pexpect.py", line 446, in __spawn\\nraise ExceptionPexpect(\\\'Error! pty.fork() failed: \\\' + str(e))\\npexpect.ExceptionPexpect: Error! pty.fork() failed: out of pty devices\')')PLz tell me what is the problem,Thanking you.Regards, ---ViNOJ DAViS--- 5, 50, 500, 5000 - Store N number of mails in your inbox. Click here. -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange varargs issue
Mike schrieb: > I'm not sure if this is a bug or if I'm just not understanding > something correctly. I'm running the following (broken.py) on > ActivePython 2.5.1.1, based on Python 2.5.1 (r251:54863 5/1/2007) as > "python broken.py foo" (on Windows, of course): > > > #!/bin/env python > > import sys > > class foobar(object): > def func(arg): > print 'foobar.func: %r' % arg This needs to be def func(self, arg): And then of course for aclling, you need an instance of foobar as first argument. Either explicit, or implicit: unbound_m = foobar.func unbound_m(some_foobar, arg) bound_m = foobar().func bound_m(arg) Or you do @classmethod def func(cls, arg): ... Then you only need one argument, but beware: it's a classmethod, not an instancemethod anymore. Diez Diez -- http://mail.python.org/mailman/listinfo/python-list
RE: pydepend (checking dependencies like jdepend) ?
No, py2exe does not display such information but has an algorithm to collect such information. Perhaps this is a starting point for you. Stefan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Bernhard Merkle Sent: Friday, January 04, 2008 2:25 PM To: python-list@python.org Subject: Re: pydepend (checking dependencies like jdepend) ? On Jan 4, 1:57 pm, "Stefan Schukat" <[EMAIL PROTECTED]> wrote: > Hi, > > try to look at py2exe. This module scans all dependencies to pack them > into one executable. my intention is to _know_ (or display or list or whatever) the dependencies. (see also my original posting). The aim is to control and have a view on modularization and e.g. avoid unnecessary bidirectional dependencies etc. does py2.exe display such information ? Berni. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Strange varargs issue
I'm not sure if this is a bug or if I'm just not understanding something correctly. I'm running the following (broken.py) on ActivePython 2.5.1.1, based on Python 2.5.1 (r251:54863 5/1/2007) as "python broken.py foo" (on Windows, of course): #!/bin/env python import sys class foobar(object): def func(arg): print 'foobar.func: %r' % arg __f = foobar() def caller(a): print 'caller: %r' % a __f.func(a) def main(): rest = sys.argv[1:] print 'main: %r' % rest caller(*rest) if __name__ == '__main__': main() ...and the result of running this ("python broken.py foo") is: main: ['foo'] caller: 'foo' Traceback (most recent call last): File "broken.py", line 21, in main() File "broken.py", line 18, in main caller(*rest) File "broken.py", line 13, in caller __f.func(a) TypeError: func() takes exactly 1 argument (2 given) How can this possibly be? The "caller" print statement obviously shows "a" is singular. Thanks in advance for any and all insight... Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes - pointer to array of structs?
[EMAIL PROTECTED] schrieb: > (Is this the right place to ask ctypes questions? There's a mailing list > but the last post to it seems to have been in November 2006.) You could use the ctypes-users mailing list: https://lists.sourceforge.net/lists/listinfo/ctypes-users It is also available via gmane. > Using ctypes I reference a structure which contains a pointer to an array of > another structure: > > class SYMBOL(Structure): > _fields_ = [("symbol", c_char_p), > ("num", c_int), > ("units", c_int), > ("baseprice", c_int), > ("active", c_int)] > SYMBOL_PTR = POINTER(SYMBOL) > > class TABLE(Structure): > _fields_ = [("map", SYMBOL_PTR), > ("nsymbols", c_uint), > ...] > > Effectively, TABLE.map is an array of TABLE.nsymbols SYMBOLS. How to I > reference elements in that array? In C I would just treat TABLE.map like an > array and index into it (for i=0; i< TABLE.nsymbols; i++) ...). This is > data returned from a C library, not something I'm building in Python to pass > into C. Assuming you got a pointer to TABLE from a function call like this: somefunction.restype = POINTER(TABLE) ptab = somefunction(...) then you should be able to use this code (C has the '*' operator to derefence pointers, Python does not so you have to use p[0] instead of *p): table = ptab[0] symp = table.map for i in range(table.nsymbols): sym = symp[0] print sym.symbol, sym.num, sym.units, sym.baseprice Thomas -- http://mail.python.org/mailman/listinfo/python-list
Fortran to Python
I got someone who asked me to make changes in an old Fortran program she is using for some calculations. The calculations are pretty standard aside from 2 calls to DLINCG (an IMSL numerical_libraries function to calculate an inverse matrix). What I wonder about, does anybody have a Fortran to Python conversion page somewhere to map some of the basic types to Python equivalents? What kind of speed difference should I expect? -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ One often hears the most when everyone is silent... -- http://mail.python.org/mailman/listinfo/python-list
pydepend (checking dependencies like jdepend) ?
Hi there, think %Subject says all. I am wondering if there is some tool to check dependencies within python programs. (something like jdepend for python ;-) Of course the dependencies are at runtime (dynamic) and not statically +dynamically (as in Java), but anyway it would be interesting to know of them (for better modularization e.g.) TIA, Berni. -- http://mail.python.org/mailman/listinfo/python-list
Details about pythons set implementation
Hi, I'm interested in details about how sets are implemented in python. They seem to be quite fast and I found some remarks who state, that the implementation is highly optimized. I need to implemented sets in C/C++ and need a starting point on how to do it right. Could somebody give me a starting point? regards, Achim -- http://mail.python.org/mailman/listinfo/python-list
Re: Fortran to Python
-On [20080104 14:22], Jeroen Ruigrok van der Werven ([EMAIL PROTECTED]) wrote: >What I wonder about, does anybody have a Fortran to Python conversion page >somewhere to map some of the basic types to Python equivalents? Just to share my own ideas: Seems COMPLEX*16/complex*16 ~= complex REAL*8/real*8 ~= float INTEGER/integer ~= int/long -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ Necessity is the mother of invention... -- http://mail.python.org/mailman/listinfo/python-list
Re: Who's to blame?
On Jan 4, 3:35 am, Nicola Musatti <[EMAIL PROTECTED]> wrote: > Hallo, Mike. > First of all, thanks to both you and Rob for your answers. I now see > that the wxPython group would have been a better place to post to, all > the more so given the tight connection between the wxPython and > wxWidgets projects, of which at first I wasn't aware. > > On Jan 3, 8:19 pm, [EMAIL PROTECTED] wrote: > [...] > > > > > I've never created a modal dialog like this. Instead, I follow the > > wxPython in Action book examples (most of the time), which would do a > > yes/no dialog like this: > > > > > > dlg = wx.MessageDialog(None, 'Some Message', 'A Message Box', > > wx.YES_NO | wx.QUESTION) > > retCode = dlg.ShowModal() > > if retCode == wx.ID_YES: > ># do something > >print 'yes' > > else: > ># do something else > >print 'no' > > dlg.Destroy() > > > > > Actually my example started out as something like > > if wx.MessageBox(message="Some message", caption="Some caption", > style=wx.YES|wx.NO) == wx.YES: > pass > > I had to change it because the actual message can become very long, so > I assembled a dialog with a scrollable text field. Maybe I'm expecting > to much of wxStdDialogButtonSizer, but I still feel that given that > both your method and mine above work straight away, it should provide > the same behaviour with Yes/No buttons as with OK/Cancel ones. > > Cheers, > Nicola Musatti Nicola, I have sub-classed wx.Dialog to do my own custom modal dialogs as well. You can use sizers and put whatever widgets you want onto it that way. Just make sure that when you create the Yes/No buttons, you give them the wx.ID_YES or wx.ID_NO ids, rather than -1 or wx.ID_ANY. yesBtn = wx.Button(parent, wx.ID_YES, 'Yes') noBtn = wx.Button(parent, wx.ID_NO, 'No') Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange varargs issue
Mike wrote: >__f.func(a) > TypeError: func() takes exactly 1 argument (2 given) > > How can this possibly be? The "caller" print statement obviously > shows "a" is singular. __f.func(a) is a method call, and methods always get the object itself as an extra initial argument. to fix this, add "self" to the method signature: class foobar(object): def func(self, arg): print 'foobar.func: %r' % arg see the tutorial for more info. > I'm not sure if this is a bug or if I'm just not understanding > something correctly. you are aware that blaming your mistakes on bugs in widely used code is somewhat rude, right? http://www.catb.org/~esr/faqs/smart-questions.html#id306617 -- http://mail.python.org/mailman/listinfo/python-list
how to build a dict including a large number of data
hi everyone i'm a newbie to python :) i have a binary file named test.dat including 960 records. the record format is int a + int b + int c + int d i want to build a dict like this: key=int a,int b values=int c,int d i choose using bsddb and it takes about 140 seconds to build the dict. what can i do if i want to make my program run faster? or is there another way i can choose? Thanks in advance. My Code: --- my_file = file('test.dat','rb') content = my_file.read() record_number = len(content) / 16 db = bsddb.btopen('test.dat.db','n',cachesize=5) for i in range(0,record_number): a = struct.unpack("",content[i*16:i*16+16]) db['%d_%d' % (a[0],a[1])] = '%d_%d' % (a[2],a[3]) db.close() my_file.close() -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange varargs issue
On Jan 4, 3:45 pm, Mike <[EMAIL PROTECTED]> wrote: > I'm not sure if this is a bug or if I'm just not understanding > something correctly. I'm running the following (broken.py) on > ActivePython 2.5.1.1, based on Python 2.5.1 (r251:54863 5/1/2007) as > "python broken.py foo" (on Windows, of course): > > #!/bin/env python > > import sys > > class foobar(object): > def func(arg): > print 'foobar.func: %r' % arg > > __f = foobar() > > def caller(a): > print 'caller: %r' % a > __f.func(a) > > def main(): > rest = sys.argv[1:] > print 'main: %r' % rest > caller(*rest) > > if __name__ == '__main__': > main() > > ...and the result of running this ("python broken.py foo") is: > > main: ['foo'] > caller: 'foo' > Traceback (most recent call last): > File "broken.py", line 21, in > main() > File "broken.py", line 18, in main > caller(*rest) > File "broken.py", line 13, in caller > __f.func(a) > TypeError: func() takes exactly 1 argument (2 given) > > How can this possibly be? The "caller" print statement obviously > shows "a" is singular. > > Thanks in advance for any and all insight... > > Mike class foobar(object): def func(arg): print 'foobar.func: %r' % arg def caller(a): __f.func() >>> main: ['foo'] >>> caller: 'foo' >>> foobar.func: <__main__.foobar object at 0x00A45550> class foobar(object): def func(self, arg): print 'foobar.func: %r' % arg def caller(a): __f.func(a) >>> main: ['foo'] >>> caller: 'foo' >>> foobar.func: 'foo' You're already passing the object as an argument in the first case. -- http://mail.python.org/mailman/listinfo/python-list
RE: pydepend (checking dependencies like jdepend) ?
Hi, try to look at py2exe. This module scans all dependencies to pack them into one executable. Stefan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Bernhard Merkle Sent: Friday, January 04, 2008 1:14 PM To: python-list@python.org Subject: pydepend (checking dependencies like jdepend) ? Hi there, think %Subject says all. I am wondering if there is some tool to check dependencies within python programs. (something like jdepend for python ;-) Of course the dependencies are at runtime (dynamic) and not statically +dynamically (as in Java), but anyway it would be interesting to know of them (for better modularization e.g.) TIA, Berni. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Who's to blame?
On Jan 4, 3:12 pm, [EMAIL PROTECTED] wrote: [...] > I have sub-classed wx.Dialog to do my own custom modal dialogs as > well. You can use sizers and put whatever widgets you want onto it > that way. Just make sure that when you create the Yes/No buttons, you > give them the wx.ID_YES or wx.ID_NO ids, rather than -1 or wx.ID_ANY. > > yesBtn = wx.Button(parent, wx.ID_YES, 'Yes') > noBtn = wx.Button(parent, wx.ID_NO, 'No') As far as I understand this should be taken care of by the XRC/ wxStdDialogButtonSizer combination. Anyway, I solved my problem by binding the following event handler to both buttons: def OnButton(self, event): if self.IsModal(): self.EndModal(event.GetId()) By the way, my assumption above appears to be supported by the fact that a call to ShowModal() on my dialog does return wx.ID_YES when I press the Yes button. I'm still a bit puzzled, but right now I can't afford to let it bother me too much. Thanks for your help! Cheers, Nicola -- http://mail.python.org/mailman/listinfo/python-list
Re: Cursors in a Loop
On Fri, 2008-01-04 at 00:03 -0800, Chris wrote: > You should bind all variables to save the pool. > > cursor = connection.cursor() > cursor.executemany("""insert into as_siebel_hosts_temp > values (:whole, :lot, :of, :bind, :variables) >""" > ,[(i,)[0] for i in hostlist] > ) > connection.commit() > connection.close() Huh? In the OP's example, the table one has one column. I'll openly admit that I don't know anything about Oracle, but that code doesn't make sense to me. Maybe you're trying to execute a multi-row insert, but that would be done with execute(), not executemany(), wouldn't it? Also, isn't "[(i,)[0] for i in hostlist]" exactly the same as "[i for i in hostlist]" which in turn is exactly the same as "hostlist"? -- Carsten Haese http://informixdb.sourceforge.net -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with global var
Hello all, Amazing :) The program is working properly now, the code is much better and I learned a bit more Python. Thank you all, guys. Bruno. 2008/1/4, Peter Otten <[EMAIL PROTECTED]>: > Bruno Ferreira wrote: > > > I wrote a very simple python program to generate a sorted list of > > lines from a squid access log file. > > Now that your immediate problem is solved it's time to look at the heapq > module. It solves the problem of finding the N largest items in a list > much more efficiently. I think the following does the same as your code: > > import heapq > > def key(record): > return int(record[4]) > > logfile = open("squid_access.log", "r") > records = (line.split() for line in logfile) > topsquid = heapq.nlargest(50, records, key=key) > > for record in topsquid: > print record[4] > > Peter > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: how to build a dict including a large number of data
wanzathe wrote: > i have a binary file named test.dat including 960 records. > the record format is int a + int b + int c + int d > i want to build a dict like this: key=int a,int b values=int c,int d > i choose using bsddb and it takes about 140 seconds to build the dict. you're not building a dict, you're populating a persistent database. storing ~7 records per second isn't that bad, really... > what can i do if i want to make my program run faster? > or is there another way i can choose? why not just use a real Python dictionary, and the marshal module for serialization? -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange varargs issue
You know, every once in a while, self really bites me. (I program in Java too much) Thanks for everyone who replied quickly. Mike wrote: >> [ a bunch of crap because I forgot self, nevermind sorry ] -- http://mail.python.org/mailman/listinfo/python-list
Re: how to build a dict including a large number of data
On Jan 4, 3:57 pm, wanzathe <[EMAIL PROTECTED]> wrote: > hi everyone > i'm a newbie to python :) > i have a binary file named test.dat including 960 records. > the record format is int a + int b + int c + int d > i want to build a dict like this: key=int a,int b values=int c,int d > i choose using bsddb and it takes about 140 seconds to build the dict. > what can i do if i want to make my program run faster? > or is there another way i can choose? > Thanks in advance. > > My Code: > --- > my_file = file('test.dat','rb') > content = my_file.read() > record_number = len(content) / 16 > > db = bsddb.btopen('test.dat.db','n',cachesize=5) > for i in range(0,record_number): > a = struct.unpack("",content[i*16:i*16+16]) > db['%d_%d' % (a[0],a[1])] = '%d_%d' % (a[2],a[3]) > > db.close() > my_file.close() my_file = file('test.dat','rb') db = bsddb.btopen('test.dat.db','n',cachesize=5) content = myfile.read(16) while content: a = struct.unpack('',content) db['%d_%d' % (a[0],a[1])] = '%d_%d' % (a[2],a[3]) content = myfile.read(16) db.close() my_file.close() That would be more memory efficient, as for speed you would need to time it on your side. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why python says "unexpected parameter 'mini.py'" for my code?
oyster <[EMAIL PROTECTED]> wrote: > The following is my pure-python wxwidgets test. It is hardly pure python since it depends on wxWindows and ctypes... > It runs and give a frame, but soon python comes up to say > "unexpected parameter > 'mini.py'" and I have to close it. > I cannot find the reason. Can somebody give me a hint to let it work > well? Thanks I tried it but it doesn't work at all on linux. I suggest you use wxPython and stop re-inventing the wheel! -- Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick -- http://mail.python.org/mailman/listinfo/python-list
relax
!guys make a brake see something really interesting and then heads again in: http://kavallaris.santorini.googlepages.com/ http://rooms.santorini.googlepages.com/ http://santorini.accommodation.googlepages.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: adding class functionality, nested scoping
On Jan 4, 10:43 am, [EMAIL PROTECTED] wrote: > Hi, Hi [...] > # Does not work: all enhanced methods only call the last wrapped originial > # method. It seems the name 'method' in the surrounding scope of the > # def _(...) function definition only refers to the last loop value(?) > def ERRONEOUS_enhance_all_methods(cls, replacement): > for methodname in cls.__dict__: > if not methodname.startswith("__"): > method = getattr(cls, methodname) > def _f(*args, **kwargs): > return replacement(method, *args, **kwargs) > _f.__name__ = methodname > setattr(cls, methodname, types.MethodType(_f, None, cls)) > This is normal: After ERRONEOUS_enhance_all_methods is called, the value method is the one from the last iteration of the loop. All subsequent references to 'method' (in function _f) will return that last value. To solve this problem you need to fix the object 'method' is bound to in function _f: def enhance_all_methods(cls, replacement): # The following binds 'method' to its current value in _f def replace(method): def _f(*args, **kwargs): return replacement(method, *args, **kwargs) return _f for methodname in cls.__dict__: if not methodname.startswith("__"): _f = replace(getattr(cls, methodname)) _f.__name__ = methodname setattr(cls, methodname, types.MethodType(_f, None, cls)) Of course this looks more like your first version, which trims down to the following and is probably a better option: def enhance_all_methods(cls, replacement): for methodname in cls.__dict__: if not methodname.startswith("__"): enhance_method(cls, methodname, replacement) HTH -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: pydepend (checking dependencies like jdepend) ?
On Jan 4, 1:57 pm, "Stefan Schukat" <[EMAIL PROTECTED]> wrote: > Hi, > > try to look at py2exe. This module scans all dependencies to pack them > into one executable. my intention is to _know_ (or display or list or whatever) the dependencies. (see also my original posting). The aim is to control and have a view on modularization and e.g. avoid unnecessary bidirectional dependencies etc. does py2.exe display such information ? Berni. -- http://mail.python.org/mailman/listinfo/python-list
pexpect - not printing more than 80 columns
Dear All, I'm trying to write a script which will check the progress of my MD-simulations on a cluster. So far I've been able to ssh to each node and retrieve the data I was looking for by sending the command ""top -c -n 1|grep mdrun" with pexpect. Unfortunately the string I'm looking for is beyond the 80th column, so I can't use it! Does anyone have a clue on how to retrieve the output of top (or anything for that matter) beyond the 80th column? Thanks in advance! Cheers, Maximilian -- http://mail.python.org/mailman/listinfo/python-list
Re: python interfaces
On Jan 4, 3:59 pm, hyperboreean <[EMAIL PROTECTED]> wrote: > Hi, > Probably it has been asked before, but I'll still ask. > Why doesn't python provide interfaces trough its standard library? Or it > was ever proposed to be included in the language? > Zope's implementation seems pretty flexible and straightforward. > > Thanks. Python 3.0 will introduce Abstract Base Classes: http://www.python.org/dev/peps/pep-3119/ -- http://mail.python.org/mailman/listinfo/python-list
How Does This Static Variable Work?
Hi; I read this example somewhere, but I don't understand it <:-) Can someone please explain how static variables work? Or recommend a good how-to? import random def randomwalk_static(last=[1]): # init the "static" var(s) rand = random.random() # init a candidate value if last[0] < 0.1: # threshhold terminator return None # end-of-stream flag while abs(last[0]-rand) < 0.4: # look for usable candidate print '*', # display the rejection rand = random.random() # new candidate last[0] = rand # update the "static" var return rand TIA, Victor -- http://mail.python.org/mailman/listinfo/python-list
Scales question
Hi all... I want to represent a point in 800 X 600 board in a 640 X 480 board..., for example (13, 50) in 640X480 to 800X600 so.. will be like this... Xscale = (13 * 800)/640 Xscale = 16.25 Yscale = (50 * 600)/480 Yscale = 62.5 what happend with the decimals??? I round up or down??? or there is another way to calculate this more accurate? Thks... -- http://mail.python.org/mailman/listinfo/python-list
python interfaces
Hi, Probably it has been asked before, but I'll still ask. Why doesn't python provide interfaces trough its standard library? Or it was ever proposed to be included in the language? Zope's implementation seems pretty flexible and straightforward. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Details about pythons set implementation
Achim Domma <[EMAIL PROTECTED]> writes: > I'm interested in details about how sets are implemented in python. > They seem to be quite fast and I found some remarks who state, that > the implementation is highly optimized. I need to implemented sets > in C/C++ and need a starting point on how to do it right. Could > somebody give me a starting point? You can simply look at the implementation, Objects/setobject.c in the Python source code. Most that it's mostly copy-paste from the dict implementation (dictobject.c) and that both are quite involved and optimized for the use by Python. They're not general implementation of sets from use in C. The "highly optimized" remarks should be understood in the context of Python, not in the context of other C and C++ set libraries. I don't know how well Python sets compare to other set libraries, but I doubt that it's much faster than the median (which "highly optimized" could be understood to imply). BTW if you're using C++, why not simply use std::set? If you need it called from C, you can wrap the needed methods in a C-accessible API. -- http://mail.python.org/mailman/listinfo/python-list
Re: Details about pythons set implementation
On Jan 4, 2008 9:54 AM, Achim Domma <[EMAIL PROTECTED]> wrote: > Hi, > > I'm interested in details about how sets are implemented in python. > They seem to be quite fast and I found some remarks who state, that > the implementation is highly optimized. I need to implemented sets in > C/C++ and need a starting point on how to do it right. Could somebody > give me a starting point? #include -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: Fortran to Python
-On [20080104 15:56], Robin Becker ([EMAIL PROTECTED]) wrote: >you probably want to look at numpy an extension that handles lots of matrix >things with great ease. I think it now lives at http://scipy.org/ Yeah, I am aware of SciPy/NumPy, but aside from these two calls to do this inverse matrix calculation the rest is just stock mathematics, supported by Python. I am trying to avoid another external dependency if I can. :) -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ Time is merely a residue of Reality... -- http://mail.python.org/mailman/listinfo/python-list
Memory Leaks and Heapy
Hi list, Firstly, this is my first post here, so I hope I'm not breaking some unwritten etiquette rule about asking questions involving several different libraries. I'm trying to plug some memory leaks in a TurboGears program. We (the Fedora Project) have a few apps in Turbogears in infrastructure that all seem to be running into the same issues in a variety of configurations. Hopefully when I get to the cause of this in one app, Smolt, we can fix the others too. The app in question is Smolt, which uses TurboGears, SQLAlchemy with a MySQL backend, and simplejson for message passing between the server and client. Smolt takes voluntary hardware reports from its clients, and generally is configured to submit around the beginning of the month. Normally, our main data is cached by some separate processes that run short term, so we don't see any rapid memory growth, except for the beginning of each month, which makes isolating the problem to a few function calls fairly simple. To watch for memory growth, I simply have a client hammer the server with 1-3 threads submitting information simultaneously, 100 times, with a few deletion operations in between. To monitor for memory leaks, I'm using Heapy. To insert Heapy into the process, instead of calling 'start_server', a cherrypy method that does what you think it does and blocks, I'm using the module 'threading' to push it into a new thread. Using the process in heapy's documentation, I find that after running a single thread, there is about 1200 bytes of leaked memory. Despite this, the python process running the server has managed to grow from 16-18MB to something between 23-28MB each time I try this. After a second iteration, heapy shows 1168 bytes leaked. If heapy is correct, this means there are not many leaked objects in the python space. Running a larger example, say 100 threads, for a total of 10k submissions takes about an hour, and in the process, python baloons up to about 48MB. Still no signs of any missing objects. 48MB is not alot relatively speaking, but no amount of waiting seems to show python giving back that memory afterwards. On our production server, we have up to 200k machines all updating their information over a 3 day period, in which the server process manages to reach 600MB before we forcefully restart it. A couple of developers have mentioned that python might be fragmenting its memory space, and is unable to free up those pages. How can I go about testing for this, and are there any known problems like this? If not, what else can I do to look for leaks? -Yaakov -- http://mail.python.org/mailman/listinfo/python-list
Re: pydepend (checking dependencies like jdepend) ?
Stefan Schukat schrieb: > No, py2exe does not display such information but has an algorithm to > collect such information. > Perhaps this is a starting point for you. If py2exe is run with the -x flag, it does display a cross-reference in a browser window. Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: how to build a dict including a large number of data
On 1月4日, 下午10时17分, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > wanzathe wrote: > > i have a binary file named test.dat including 960 records. > > the record format is int a + int b + int c + int d > > i want to build a dict like this: key=int a,int b values=int c,int d > > i choose using bsddb and it takes about 140 seconds to build the dict. > > you're not building a dict, you're populating a persistent database. > storing ~7 records per second isn't that bad, really... > > > what can i do if i want to make my program run faster? > > or is there another way i can choose? > > why not just use a real Python dictionary, and the marshal module for > serialization? > > hi,Fredrik Lundn you are right, i'm populating a persistent database. i plan to use a real Python dictionary and use cPickle for serialization at first, but it did not work because the number of records is too large. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Fortran to Python
Jeroen Ruigrok van der Werven wrote: > -On [20080104 14:22], Jeroen Ruigrok van der Werven ([EMAIL PROTECTED]) wrote: >> What I wonder about, does anybody have a Fortran to Python conversion page >> somewhere to map some of the basic types to Python equivalents? > > Just to share my own ideas: > > Seems > > COMPLEX*16/complex*16 ~= complex > REAL*8/real*8 ~= float > INTEGER/integer ~= int/long > you probably want to look at numpy an extension that handles lots of matrix things with great ease. I think it now lives at http://scipy.org/ -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
RE: dictionary/hash and '1' versus 1
> From: Stephen Hansen [mailto:[EMAIL PROTECTED] > Sent: Thursday, January 03, 2008 7:39 PM > To: Reedick, Andrew > Cc: python-list@python.org > Subject: Re: dictionary/hash and '1' versus 1 > > > > Well one important thing to learn while learning Python is that while the > language is dynamically typed-- it is also /strongly/ typed. Every piece > of data has an explicit type and it doesn't change unless you change it. Meh, mixing dynamic and strong typing is a mixed blessing. You don't find out that you screwed up the data types until the code block is actually executed. Reminds me of Nostradamus. He may have predicted the future[1], but the predictions are so vague/convoluted that you can only figure out the predictions in hindsight. > It relies on duck typing a lot, and doesn't care if you mix and match > (even partially) compatible types as long as the operations are there, > but one type will always be distinct and remain that type until you > explicitly convert it. > > A single integer is distinctly different from a sequence of characters in > some encoding that may just happen to contain representations of a > number so they'll hash differently :) Depends on the context. The machine encoding may be different, but in human terms they "should" be the same. Perl managed to achieve an impressive blend of presenting data as human friendly or as machine bits when it made sense to do so. So much so, that Perl is probably the only language I've used that will do what you mean instead of what you say. Nice, but frightening in some ways. > One type will basically never implicitly convert into another type. > > To me, this sounds like the function should have converted the type > explicitly on return. Or maybe you need to convert it explicitly on > receipt. Type casting is easy, IFF you remember to do so. The problem was that I missed the fact that one (important) function was returning a string instead of an int, and since Python supports heterogenous data structures, the human has to remember to keep the key's data type homongenous. That and Perl does so much automatic type conversion in such a sensible way, that I stopped worrying about mixing data types, which is making the Python transition a tad more error prone. Because of Perl, I almost consider automatic type casting to be the next "you don't have to manage your own memory" that people loved about Java. =O > But if you are in a use-case where you really don't care and only > want to hash strings, you can create a dict subclass easily that > overrides __setitem__ to always str() the input. Check out the > UserDict class. UserDict looks like it could be useful. Thanks for the info. > A similar method lets you make 'case-insensitive' dicts, for example. > > Were such a thing to happen automagically, you could get some > weird situations, such as "assert (key in dict) == (key in dict.keys())" > failing. I'm assuming that you would just need to overload the 'in' operator and .keys() method to be case insensitive also. [1] No, he didn't. * The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers. GA625 -- http://mail.python.org/mailman/listinfo/python-list
Re: Pivot Table/Groupby/Sum question
Petr thanks so much for your input. I'll try to learn SQL, especially if I'll do a lot of database work. I tried to do it John's way as en exercise and I'm happy to say I understand a lot more. Basically I didn't realize I could nest dictionaries like db = {country:{genre:{sub_genre:3}}} and call them like db[country][genre][sub_genre]. The Python Cookbook was quite helpful to figure out why items needed to be added the way they did. Also using the structure of the dictionary was a conceptually easier solution than what I found on http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/334695. So, now I need to work on writing it to Excel. I'll update with the final code. Thanks again. #Movie Store Example class PivotData: def __init__(self): self.total_mov = 0 self.total_cou = {} self.total_gen = {} self.total_sub = {} self.total_cou_gen ={} self.db = {} def add_data(self,country,genre,sub_genre,value): self.total_mov += value try: self.total_cou[country] += value except KeyError: self.total_cou[country] = value try: self.total_gen[genre] += value except: self.total_gen[genre] = value try: self.total_sub[sub_genre] += value except: self.total_sub[sub_genre] = value try: self.total_cou_gen[country][genre] += value except KeyError: try: self.total_cou_gen[country][genre] = value except KeyError: self.total_cou_gen[country] = {genre:value} try: self.db[country][genre][sub_genre] += value except KeyError: try: self.db[country][genre][sub_genre] = value except KeyError: try: self.db[country][genre] = {sub_genre:value} except: self.db[country] = {genre:{sub_genre:value}} data = [['argentina','Horror', 'Slasher',4], ['argentina','Horror', 'Halloween',6], ['argentina','Drama','Romance',5], ['argentina','Drama','Romance',1], ['argentina','Drama','True Life',1], ['japan','Classics','WWII',1], ['japan','Cartoons','Anime',1], ['america','Comedy','Stand-Up',1], ['america','Cartoons','WB',10], ['america','Cartoons','WB',3]] COUNTRY, GENRE, SUB_GENRE, VALUE =range(4) x=PivotData() for s in data: x.add_data(s[COUNTRY],s[GENRE],s[SUB_GENRE],s[VALUE]) print print 'Total Movies:\n', x.total_mov print 'Total for each country\n', x.total_cou print 'Total Genres\n', x.total_gen print 'Total Sub Genres\n', x.total_sub print 'Total Genres for each Country\n', x.total_cou_gen print print x.db -- http://mail.python.org/mailman/listinfo/python-list
Re: What's the limit of variables size in pyhton?
In the darkest hour on Mon, 31 Dec 2007 20:53:28 -0200, Gabriel Genellina <[EMAIL PROTECTED]> screamed: >> Is that mean that i can deal with files with size more than 2GB only >> if the available memory allow > > To be more precise, that depends on the OS. On Windows there is a limit of > 2GB adressable memory per process (this is unrelated to the amount of > physical memory). That's the 32bit Windows limit only (afaik). -- [ Artur M. Piwko : Pipen : AMP29-RIPE : RLU:100918 : From == Trap! : SIG:240B ] [ 16:51:04 user up 11576 days, 4:46, 1 user, load average: 0.97, 0.71, 0.01 ] No wonder people are so horrible when they start life as children. -- K. Amis -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory Leaks and Heapy
-On [20080104 16:11], Yaakov Nemoy ([EMAIL PROTECTED]) wrote: >I'm trying to plug some memory leaks in a TurboGears program. We (the >Fedora Project) have a few apps in Turbogears in infrastructure that >all seem to be running into the same issues in a variety of >configurations. Hopefully when I get to the cause of this in one app, >Smolt, we can fix the others too. [snip] >A couple of developers have mentioned that python might be fragmenting >its memory space, and is unable to free up those pages. How can I go >about testing for this, and are there any known problems like this? >If not, what else can I do to look for leaks? As various people pointed out to me: http://wingolog.org/archives/2007/11/27/reducing-the-footprint-of-python-applications That might help. Aside from that (rant), I seriously dislike Python's memory management and even more the fairly arcane ways people have to go about debugging/troubleshooting some 600 MB to 2-3 GB(!) of resident memory use by Python. Personally I consider this the weakest point of Python. Given the fact there is a garbage collector this sort of keeping track of memory seems a bit contradictory to the concept of garbage collection. (And yes, I am investigating this and also how to get it leaner and better.) -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ Speak the sweet truth... -- http://mail.python.org/mailman/listinfo/python-list
Re: how to use bool
On Jan 3, 10:09 am, [EMAIL PROTECTED] wrote: > > With my philosophical programming hat on the first thing I'd say (as a > fairly beginning python programmer) is "avoid multiple returns from a > function/method if at all possible". They breed all sorts of problems > and errors, in particular if there's any clearing up to do you have to > do it in lots of places (or you forget it in some places). > This conventional wisdom predates the introduction of constructs such as try-catch-finally. In fact, you are just lulling yourself into some false security if you think that that single return at the end of your method is the only exit point of your routine. Exceptions can (and will) happen just about anywhere. I know, you had your C hat on, but even C++'ers fall into this trap. I was on a project that cited explicit delete statements during code reviews as likely memory leaks in the face of exceptions, and each was to be replaced with auto-cleanup objects such as auto_ptr. The same was done for other resource alloc/release pairs, such as locks, database connections, cursors, etc. These were long-running server processes, and they ran for months with no memory growth at all. If I were to suggest a similar topic for Python code reviews, it would be to look at paired statements and to ensure that the cleanup/ recovery statement was wrapped in a finally block. There's more than just memory that needs bookkeeping, so garbage collection wont solve all these other resource management problems. -- Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: how to use bool
On Jan 3, 7:49 am, [EMAIL PROTECTED] wrote: > hi, i have some code where i set a bool type variable and if the value > is false i would like to return from the method with an error msg.. > being a beginner I wd like some help here > > class myclass: > . > def mymethod(self): > success=True > msg="all validation OK" > success=validateSthing() > if(success==False): >msg="sthing failed" >return (success,msg) > > dosomeprocessing() > . > success=validateSthingelse() > if(success==False): >msg="sthingelse failed" >return (success,msg) > domoreprocessing() > >return(success,msg) > > i would like to know if this way of doing this is OK..I have need of > many kinds of validations in this ..is there a better way of doing > this ? > > thank you class SthingError(Exception): def __init__(self, success, msg): class myclass: . def mymethod(self): success=True if not validateSthing(): msg="sthing failed" return (success,msg) dosomeprocessing() . if not validateSthingelse(): msg="sthingelse failed" return (success,msg) domoreprocessing() return(success,"all validation OK") -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory Leaks and Heapy
On Jan 4, 2008 11:10 AM, M.-A. Lemburg <[EMAIL PROTECTED]> wrote: > If you're using lots of small objects, you may be running into a > problem with the Python memory allocation mechanism, pymalloc. It used > to not return memory to the system. In Python 2.5 (IIRC, could be > 2.6) this was changed to at least return completely empty blocks > back to the OS. For details, see Objects/obmalloc.c The most common answer I heard was possible fragmentation, meaning there are no or few completely empty blocks to be found. If there are no 'leaks' in the VM, then it's probably related to how memory is freed. > This could be caused by interned strings which are kept in a special > pool dictionary to speed up string comparisons. That's quite possible, a majority of the code is a huge number of SQL connections and code. All string based. > However, the first thing to check is whether any of the C extension > modules you are using is leaking memory. Python itself is usually > well tested for memory leaks, but this is less so for C extension > modules and it's easy to mis a few Py_DECREFs (decrementing a > Python object's reference count), causing objects to live forever. I'll try to track it down, but AFAIK, most of the code is python, and the only C code there would be is the MySQL container. How can I debug the VM though, to determine where the leak lies? Heapy wasn't able to tell me this, and this is the important aspect. I'm wondering how most people go about determining the causes of leaks like these, so I can provide some accurate bug information. -Yaakov -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory Leaks and Heapy
On 2008-01-04 16:07, Yaakov Nemoy wrote: > Hi list, > > Firstly, this is my first post here, so I hope I'm not breaking some > unwritten etiquette rule about asking questions involving several > different libraries. > > I'm trying to plug some memory leaks in a TurboGears program. We (the > Fedora Project) have a few apps in Turbogears in infrastructure that > all seem to be running into the same issues in a variety of > configurations. Hopefully when I get to the cause of this in one app, > Smolt, we can fix the others too. > > The app in question is Smolt, which uses TurboGears, SQLAlchemy with a > MySQL backend, and simplejson for message passing between the server > and client. Smolt takes voluntary hardware reports from its clients, > and generally is configured to submit around the beginning of the > month. Normally, our main data is cached by some separate processes > that run short term, so we don't see any rapid memory growth, except > for the beginning of each month, which makes isolating the problem to > a few function calls fairly simple. To watch for memory growth, I > simply have a client hammer the server with 1-3 threads submitting > information simultaneously, 100 times, with a few deletion operations > in between. To monitor for memory leaks, I'm using Heapy. > > To insert Heapy into the process, instead of calling 'start_server', a > cherrypy method that does what you think it does and blocks, I'm using > the module 'threading' to push it into a new thread. Using the > process in heapy's documentation, I find that after running a single > thread, there is about 1200 bytes of leaked memory. Despite this, the > python process running the server has managed to grow from 16-18MB to > something between 23-28MB each time I try this. After a second > iteration, heapy shows 1168 bytes leaked. If heapy is correct, this > means there are not many leaked objects in the python space. Running > a larger example, say 100 threads, for a total of 10k submissions > takes about an hour, and in the process, python baloons up to about > 48MB. Still no signs of any missing objects. > > 48MB is not alot relatively speaking, but no amount of waiting seems > to show python giving back that memory afterwards. On our production > server, we have up to 200k machines all updating their information > over a 3 day period, in which the server process manages to reach > 600MB before we forcefully restart it. > > A couple of developers have mentioned that python might be fragmenting > its memory space, and is unable to free up those pages. How can I go > about testing for this, and are there any known problems like this? > If not, what else can I do to look for leaks? If you're using lots of small objects, you may be running into a problem with the Python memory allocation mechanism, pymalloc. It used to not return memory to the system. In Python 2.5 (IIRC, could be 2.6) this was changed to at least return completely empty blocks back to the OS. For details, see Objects/obmalloc.c This could be caused by interned strings which are kept in a special pool dictionary to speed up string comparisons. However, the first thing to check is whether any of the C extension modules you are using is leaking memory. Python itself is usually well tested for memory leaks, but this is less so for C extension modules and it's easy to mis a few Py_DECREFs (decrementing a Python object's reference count), causing objects to live forever. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 04 2008) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 -- http://mail.python.org/mailman/listinfo/python-list
Re: choosing random dynamic port number
On 2008-01-04, Giampaolo Rodola' <[EMAIL PROTECTED]> wrote: >> def GenerateDynamicPortNumber(): >> return 0 >> >> (to get the actual number, use getsockname() on the socket after you've >> called "bind" on it) >> >> > > By using 0 as port number value you let kernel choose a free > unprivileged random port: The port number chosen isn't random on many OSes. If the OP really wants a random port number, he'll have to generate it himself. -- Grant Edwards grante Yow! Look! A ladder! at Maybe it leads to heaven, visi.comor a sandwich! -- http://mail.python.org/mailman/listinfo/python-list
Re: How Does This Static Variable Work?
On Jan 4, 2008 10:17 AM, Victor Subervi <[EMAIL PROTECTED]> wrote: > Hi; > I read this example somewhere, but I don't understand it <:-) Can someone > please explain how static variables work? Or recommend a good how-to? > > > import random > > def randomwalk_static(last=[1]): # init the "static" var(s) > > rand = random.random() # init a candidate value > Simulating C's static local variables is the (in)famous application for this case of optimization in Python's design. Consult the following entry in the Python General Programming FAQ for further information. http://www.python.org/doc/faq/general/#why-are-default-values-shared-between-objects -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: how to use bool
Chris Mellon <[EMAIL PROTECTED]> wrote: > On 03 Jan 2008 16:09:53 GMT, <[EMAIL PROTECTED]> wrote: > > > > [EMAIL PROTECTED] wrote: > > > hi, i have some code where i set a bool type variable and if the value > > > is false i would like to return from the method with an error msg.. > > > being a beginner I wd like some help here > > > > > > class myclass: > > > . > > > def mymethod(self): > > > success=True > > > msg="all validation OK" > > > success=validateSthing() > > > if(success==False): > > >msg="sthing failed" > > >return (success,msg) > > > > > > dosomeprocessing() > > > . > > > success=validateSthingelse() > > > if(success==False): > > >msg="sthingelse failed" > > >return (success,msg) > > > domoreprocessing() > > > > > >return(success,msg) > > > > > > i would like to know if this way of doing this is OK..I have need of > > > many kinds of validations in this ..is there a better way of doing > > > this ? > > > > > With my philosophical programming hat on the first thing I'd say (as a > > fairly beginning python programmer) is "avoid multiple returns from a > > function/method if at all possible". They breed all sorts of problems > > and errors, in particular if there's any clearing up to do you have to > > do it in lots of places (or you forget it in some places). > > > > This advice is highly controversial, and in the presence of exceptions > it is, at best, voodoo coding. Since your function can exit at any > point whether you do it intentionally or not, if you have crucial > cleanup it's best to write your code in a way that does it correctly > even if you return early. Following this style also often leads to odd > contortions, like extra layers of indentation, and a proliferation of > temporary flags and value-holders that aren't necessary if you write > the code in a more straight forward manner. > OK, I agree, I had my C hat on (no exceptions). On the other hand if you end with lots of levels of indentation going this way it suggests to me that maybe breaking up into more functions would be a good idea. > Make your decisions on a case by case basis of complexity, > readability, and reliability instead of following pronouncements from > on high (especially decades old pronouncements made in a different > context). Forcing a single return site in the code below adds > complexity, arguable harms readability, and provides *zero* benefit in > the code at hand. > > > So:- > > > > def mymethod(self): > > msg="sthing failed" > > success=validateSthing() > > if success: > > dosomeprocessing() > > . > > success=validateSthingelse() > > if success: > > domoreprocessing() > > > > msg="all validation OK" > > return (success,msg) > > > > I've lost the different messages for different errors but you get the > > idea. > > > > > > "if success:" rather than "if (success==True)", more readable. For > > the opposite "if not success:". > > > > > > > > -- > > Chris Green > > > > -- > > http://mail.python.org/mailman/listinfo/python-list > > -- Chris Green -- http://mail.python.org/mailman/listinfo/python-list
problem in importing .pyd
hello group , I have build a python c extension. Using python 2.5 , VS.Net 2005 on Win server 2003. But when i am trying to imort this .pyd file into python interperter or my project source code . Code compilation as well as interpreter fails. Resulting in c/c++ runtime error "R6034". The description says " An application attempted to load a c runtime library without using manifest" What should i do to resolve this problem. Looking forward to your suggestions. Thank You Abhishek -- http://mail.python.org/mailman/listinfo/python-list
Re: problem in importing .pyd
-On [20080104 16:41], abhishek ([EMAIL PROTECTED]) wrote: >What should i do to resolve this problem. Perhaps the hints/tips from http://blogs.msdn.com/nikolad/articles/427101.aspx might help? -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ I realise that nothing's as it seems... -- http://mail.python.org/mailman/listinfo/python-list
Re: How Does This Static Variable Work?
Thanks. I'll study that. Victor On Jan 4, 2008 12:34 PM, Neil Cerutti <[EMAIL PROTECTED]> wrote: > On Jan 4, 2008 10:17 AM, Victor Subervi <[EMAIL PROTECTED]> wrote: > > > Hi; > > I read this example somewhere, but I don't understand it <:-) Can > > someone please explain how static variables work? Or recommend a good > > how-to? > > > > > > import random > > > > def randomwalk_static(last=[1]): # init the "static" var(s) > > > > rand = random.random() # init a candidate value > > > > Simulating C's static local variables is the (in)famous application for > this case of optimization in Python's design. > > Consult the following entry in the Python General Programming FAQ for > further information. > > > http://www.python.org/doc/faq/general/#why-are-default-values-shared-between-objects > > -- > Neil Cerutti > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
PyGTK, libglade, and signal_autoconnect
I was having a real hard time trying to accomplish something. I couldn't find a way to automatically connect the "close" button (clicked signal) of a GUI app I was working on, to the gtk.main_quit() function. I had entered this handler directly with the GLADE-3 designer (I DON'T WANT TO USE A DICTIONARY, because I think that, using GLADE, I should be able to shortcut that procedure) but in my class the self.wtree.signal_autoconnect(self) wasn't working. Of course, I thought, it's looking for gtk.main_quit() which is obviously somewhere else. So, the only solution I could think of was to add: self.wtree.signal_autoconnect(gtk) --- having already imported gtk, naturally! This automagically worked. What I would like to know is if this is legitimate, and the right way to go. It is basically a solution I cooked up instinctively, which happened to work, but perhaps I am unaware of some other more "orthodox" way of doing it. Thank you. Rock -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory Leaks and Heapy
Jeroen Ruigrok van der Werven wrote: > Aside from that (rant), I seriously dislike Python's memory management and > even more the fairly arcane ways people have to go about > debugging/troubleshooting some 600 MB to 2-3 GB(!) of resident memory use by > Python. > > Personally I consider this the weakest point of Python. Given the fact there > is a garbage collector this sort of keeping track of memory seems a bit > contradictory to the concept of garbage collection. You are welcome to join the team and improve the memory management. We are constantly looking for new developers and way to enhance Python. If you feel so strong about the memory management please provide patches and benchmarks. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory Leaks and Heapy
On Jan 4, 2008 10:34 AM, Jeroen Ruigrok van der Werven <[EMAIL PROTECTED]> wrote: > As various people pointed out to me: > http://wingolog.org/archives/2007/11/27/reducing-the-footprint-of-python-applications It did; it's what lead me to Heapy. > Aside from that (rant), I seriously dislike Python's memory management and > even more the fairly arcane ways people have to go about > debugging/troubleshooting some 600 MB to 2-3 GB(!) of resident memory use by > Python. I agree, but in many cases, the language is powerful enough, that we're just waiting for the implementation to catch up. > Personally I consider this the weakest point of Python. Given the fact there > is a garbage collector this sort of keeping track of memory seems a bit > contradictory to the concept of garbage collection. There are examples of how even the best garbage collection can still miss a few details, mainly because certain objects are holding on to information they don't actually need, and the documentation for Heapy even demonstrates an example in Tkinter. -Yaakov -- http://mail.python.org/mailman/listinfo/python-list
Re: import zlib in 2.5 fails
On Jan 4, 2:19 am, stuntgoat <[EMAIL PROTECTED]> wrote: > import zlib works in Python 2.4 (debian etch AMD64 - default python > version for that distro) > > I built python 2.5 from source; zlib is not importable. 2.5 has been available for some time in the Debian repositories. Installing the .deb may provide better results. Note that you might also have to upgrade the dependencies to a newer version. apt-get will do all of this for you - "man apt-get" for all of the details. http://packages.debian.org/etch/python2.5 -- http://mail.python.org/mailman/listinfo/python-list
Tabnanny errors when Migrating Python 2.4 code to 2.5
Hi, When Python 2.5 first came out, I eagerly downloaded it and immediately had issues with getting it to run my 2.4 code. So I just stuck to 2.4. However, I decided this week that I really should try to get 2.5 to work. Does anyone know why code that works perfectly for months in a 2.4 environment throws indentation errors in 2.5? And why does 2.5 default to indenting 8 characters instead of 4 in IDLE? That's just weird. When I did a select all and did a Tabify region, it indented everything 8 characters then too. I haven't changed the configuration at all. Any ideas? Thanks, Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: python interfaces
hyperboreean <[EMAIL PROTECTED]> wrote: >Why doesn't python provide interfaces trough its standard library? Because they're pointless. Java interfaces are a hack around the complexities of multiple inheritence. Python does multiple inheritence Just Fine (give or take the subtleties of super()) so does not need them. Interfaces used purely with the idea of type safety provide precious little gain for the added clutter and inconvenience. Compare them to Java's requirement for explicit declaration of exceptions thrown in a method signature, if you will. -- \S -- [EMAIL PROTECTED] -- http://www.chaos.org.uk/~sion/ "Frankly I have no feelings towards penguins one way or the other" -- Arthur C. Clarke her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump -- http://mail.python.org/mailman/listinfo/python-list
Re: Details about pythons set implementation
Hrvoje Niksic <[EMAIL PROTECTED]> wrote: >BTW if you're using C++, why not simply use std::set? Because ... how to be polite about this? No, I can't. std::set is crap. The implementation is a sorted sequence -- if you're lucky, this is a heap or a C array, and you've got O(log n) performance. But the real killer is that requirement for a std::set is that T::operator< exists. Which means, for instance, that you can't have a set of complex numbers -- \S -- [EMAIL PROTECTED] -- http://www.chaos.org.uk/~sion/ "Frankly I have no feelings towards penguins one way or the other" -- Arthur C. Clarke her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump -- http://mail.python.org/mailman/listinfo/python-list
Re: how to use bool
On Jan 4, 8:51 am, bukzor <[EMAIL PROTECTED]> wrote: > On Jan 3, 7:49 am, [EMAIL PROTECTED] wrote: > > > > > hi, i have some code where i set a bool type variable and if the value > > is false i would like to return from the method with an error msg.. > > being a beginner I wd like some help here > > > class myclass: > > . > > def mymethod(self): > > success=True > > msg="all validation OK" > > success=validateSthing() > > if(success==False): > >msg="sthing failed" > >return (success,msg) > > > dosomeprocessing() > > . > > success=validateSthingelse() > > if(success==False): > >msg="sthingelse failed" > >return (success,msg) > > domoreprocessing() > > > >return(success,msg) > > > i would like to know if this way of doing this is OK..I have need of > > many kinds of validations in this ..is there a better way of doing > > this ? > > > thank you Please ignore my previous post. I accidentally submitted early. I made your example runnable (hope you don't mind). The 'pythonic' way to do error handling is to use exceptions. Below is a pretty good example. Also I've elimiated all your temporary variables. Your original function is quite short now and does the same thing. You can expand out the derived exception class if you want to pass more data to your error handler, but generally the message is enough, and that comes with the default constructor. #helper functions from time import time from random import seed seed(time()) def random(chance): from random import uniform if uniform(0, 1) < chance: return True else: return False def dosomeprocessing(x): if random(.1): raise Exception("Something bad happened while processing %s!" % x) else: print x def validateSthing(): if random(.2): raise SthingError("this Sthing is messed up!") #rewrite of your example class SthingError(Exception): pass class myclass: def mymethod(self): validateSthing() dosomeprocessing(1) validateSthing() dosomeprocessing(2) #exercise of the class and error handling m = myclass() try: m.mymethod() print "Completed successfully!" except SthingError, ste: print "STHINGERROR:" print ste except Exception, e: print e print print "This time no error handling:" m.mymethod() -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory Leaks and Heapy
On 2008-01-04 17:23, Yaakov Nemoy wrote: > On Jan 4, 2008 11:10 AM, M.-A. Lemburg <[EMAIL PROTECTED]> wrote: >> If you're using lots of small objects, you may be running into a >> problem with the Python memory allocation mechanism, pymalloc. It used >> to not return memory to the system. In Python 2.5 (IIRC, could be >> 2.6) this was changed to at least return completely empty blocks >> back to the OS. For details, see Objects/obmalloc.c > > The most common answer I heard was possible fragmentation, meaning > there are no or few completely empty blocks to be found. If there are > no 'leaks' in the VM, then it's probably related to how memory is > freed. You can check for this by using a special build of Python with disabled PyMalloc - Python will then use the standard OS malloc for all object allocations. It still uses free lists for a couple of object types, but those won't cause major leak problems. Alternatively, try to tune the PyMalloc implementation (see objmalloc.c), e.g. enable WITH_MEMORY_LIMITS. >> This could be caused by interned strings which are kept in a special >> pool dictionary to speed up string comparisons. > > That's quite possible, a majority of the code is a huge number of SQL > connections and code. All string based. Note that strings are normally *not* interned. However, you can write code in a way that causes Python to intern more strings than needed, e.g. if you dynamically compile code in your app, which then causes all identifiers in the compiled code to be interned. The interned dictionary is not exposed in Python, but you can access it using a debugger via Objects/stringobject.c:interned. >> However, the first thing to check is whether any of the C extension >> modules you are using is leaking memory. Python itself is usually >> well tested for memory leaks, but this is less so for C extension >> modules and it's easy to mis a few Py_DECREFs (decrementing a >> Python object's reference count), causing objects to live forever. > > I'll try to track it down, but AFAIK, most of the code is python, and > the only C code there would be is the MySQL container. How can I > debug the VM though, to determine where the leak lies? Heapy wasn't > able to tell me this, and this is the important aspect. I'm wondering > how most people go about determining the causes of leaks like these, > so I can provide some accurate bug information. Building Python in debug mode provides some help with this. You can then detect whether objects get properly freed. Doing explicit garbage collection via the gc module also helps in narrowing down the leak. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 04 2008) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 -- http://mail.python.org/mailman/listinfo/python-list
Re: dictionary/hash and '1' versus 1
> > > A single integer is distinctly different from a sequence of characters > in > > some encoding that may just happen to contain representations of a > > number so they'll hash differently :) > >Depends on the context. The machine encoding may be different, but > in human terms they "should" be the same. Perl managed to achieve an > impressive blend of presenting data as human friendly or as machine bits > when it made sense to do so. So much so, that Perl is probably the only > language I've used that will do what you mean instead of what you say. > Nice, but frightening in some ways. Frightening is the word I'd use-- nice is not ;-) For me, the thought that "assert (1 == '1') is True" should pass is alien and alarming to the point of rising to, "That's obviously wrong." Of course, I know other languages do things their own way .. so to each their own :) Python's philosophy of "explicit is better then implicit" is one that I appreciate and have found to be a saving grace in maintaining huge systems over the last few years. I could so see myself learning perl and sending the exact inverse message to a perl mailing list; I actually /have/ code which relies upon the exact case of numbers and strings never being implicitly converted to compare to each-other. :) Different starting point in mindset, is all. Type casting is easy, IFF you remember to do so. The problem was > that I missed the fact that one (important) function was returning a string > instead of an int, and since Python supports heterogenous data structures, > the human has to remember to keep the key's data type homongenous. > This is true; there does need to be a more aware/complete understanding of the input/output API of the code you're using. That and Perl does so much automatic type conversion in such a > sensible way, that I stopped worrying about mixing data types, which is > making the Python transition a tad more error prone. Because of Perl, I > almost consider automatic type casting to be the next "you don't have to > manage your own memory" that people loved about Java. =O Heathen. ;-) I prescribe meditation, green leaf tea, and continual chanting of the sacred scroll provided by "import this" until this unwholesome adoration of black magic practices has been cleansed from your mind. In all seriousness, I can see the appeal; but prefer the strongly typed world. Partly because of where I use it, when we do a LOT of CORBA communication (which is massively strongly typed in and of itself), protocols and careful data management/conversion. Partly habit. I don't ever worry about mixing data types; I'm just aware of what type things are (although often in vague trusting ways such as 'I am sure you are a file; please do not disappoint me'). In the end I highly doubt requiring explicit conversions has a notable negative on productivity or makes the code/logic much bigger or more complicated... whereas not having to manage one's own memory is a fairly huge savings :) > A similar method lets you make 'case-insensitive' dicts, for example. > > > > Were such a thing to happen automagically, you could get some > > weird situations, such as "assert (key in dict) == (key in dict.keys())" > > failing. > >I'm assuming that you would just need to overload the 'in' operator > and .keys() method to be case insensitive also. > For my case-insensitive dict I didn't need to modify keys at all. I wasn't interested in it being "case-preserving"; all keys were lower cased on insert(via .update, __setitem__, or .setdefault), and the 'check' value when testing or fetching was lowercased for comparison (via __contains__, __getitem__, and .get) For example, a less functional version: class caseinsensitivedict(dict): def __contains__(self, key): return dict.__contains__(self, key.lower()) def __getitem__(self, key): return dict.__getitem__(self, key.lower()) def __setitem__(self, key, value): return dict.__setitem__(self, key.lower(), value) >>> cid = caseinsensitivedict() >>> cid['This'] = 5 >>> cid {'this': 5} >>> cid.keys() ['this'] >>> print 'THIS' in cid: True I could do a case-preserving one, but it wasn't needed for the solution. --Stephen -- http://mail.python.org/mailman/listinfo/python-list
Re: Pivot Table/Groupby/Sum question
On Jan 4, 4:55 pm, [EMAIL PROTECTED] wrote: > Petr thanks so much for your input. I'll try to learnSQL, especially > if I'll do a lot of database work. > > I tried to do it John's way as en exercise and I'm happy to say I > understand a lot more. Basically I didn't realize I could nest > dictionaries like db = {country:{genre:{sub_genre:3}}} and call them > like db[country][genre][sub_genre]. The Python Cookbook was quite > helpful to figure out why items needed to be added the way they did. > Also using the structure of the dictionary was a conceptually easier > solution than what I found > onhttp://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/334695. > > So, now I need to work on writing it to Excel. I'll update with the > final code. > Hi, good to know you have succeded. I think it is matter of taste which way to go (dictionary or database). My feelig is: for data use database! If you are trying to work with data, you will need it sooner or later anyway. Again: database is made for data! :-) Writing your data to excel? Just save your numbers separated by commas in the file with the extension csv (my_data.csv) and you can open it directly in Excel. Good luck :-) Petr -- http://mail.python.org/mailman/listinfo/python-list
Re: Details about pythons set implementation
On Jan 4, 9:08 am, Sion Arrowsmith <[EMAIL PROTECTED]> wrote: > Hrvoje Niksic <[EMAIL PROTECTED]> wrote: > > >BTW if you're using C++, why not simply use std::set? > > Because ... how to be polite about this? No, I can't. std::set is > crap. The implementation is a sorted sequence -- if you're lucky, > this is a heap or a C array, and you've got O(log n) performance. > But the real killer is that requirement for a std::set is that > T::operator< exists. Which means, for instance, that you can't > have a set of complex numbers > > -- > \S -- [EMAIL PROTECTED] --http://www.chaos.org.uk/~sion/ >"Frankly I have no feelings towards penguins one way or the other" > -- Arthur C. Clarke >her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump Why cant you implement < for complex numbers? Maybe I'm being naive, but isn't this the normal definition? a + bi < c + di iff sqrt(a**2 + b**2) < sqrt(c**2, d**2) How do you implement a set without sorting? Are you expecting better than O(log n)? --Buck -- http://mail.python.org/mailman/listinfo/python-list
matplotlib fill command on axes plot problem
I am trying to use the fill command to draw a box around an object in an image. I can get the box drawn, but there seems to be a side effect. The fill command is adding white lines to the top and sometimes right side of my axes plots. I am using small images, 124x200, and after the fill command the axes plot is changed to 140x200, with the top 16 lines being all white. I am running it in interactive mode from the python command line. Here is code that reproduces the problem. from pylab import * im_array = zeros((124,200)) ion() figure() imshow(im_array) hold(True) x=40 y=40 fill([x-5,x+5,x+5,x-5],[y+5,y+5,y-5,y-5],ec='r', fill = False) hold(False) Any thoughts on what I might be doing wrong? -- http://mail.python.org/mailman/listinfo/python-list
Re: Details about pythons set implementation
On Jan 4, 5:08 pm, Sion Arrowsmith <[EMAIL PROTECTED]> wrote: [...] > But the real killer is that requirement for a std::set is that > T::operator< exists. Which means, for instance, that you can't > have a set of complex numbers This is really OT but IIRC, std::set is actually std::set< T, std::less > So make your own less_complex() function (for example, use lexicographic order), std::set should give you a set of complex numbers (sorry if some syntax is incorrect I haven't done any C++ for a while :) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Simple calculation error
Dear all, I observed a strange calculation answer, on both python 2.3.4 and 2.4.4 >>> print 753343.44 - 753361.89 -18.450001 >>> print ( (753361.89*100) - (753343.44*100) ) / 100 18.45 Can somebody help me to play correctly with decimal values? Thanks in advance, Francois Liot -- http://mail.python.org/mailman/listinfo/python-list
Re: Details about pythons set implementation
bukzor schrieb: > On Jan 4, 9:08 am, Sion Arrowsmith <[EMAIL PROTECTED]> > wrote: >> Hrvoje Niksic <[EMAIL PROTECTED]> wrote: >> >>> BTW if you're using C++, why not simply use std::set? >> Because ... how to be polite about this? No, I can't. std::set is >> crap. The implementation is a sorted sequence -- if you're lucky, >> this is a heap or a C array, and you've got O(log n) performance. >> But the real killer is that requirement for a std::set is that >> T::operator< exists. Which means, for instance, that you can't >> have a set of complex numbers >> >> -- >> \S -- [EMAIL PROTECTED] --http://www.chaos.org.uk/~sion/ >>"Frankly I have no feelings towards penguins one way or the other" >> -- Arthur C. Clarke >>her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump > > Why cant you implement < for complex numbers? Maybe I'm being naive, > but isn't this the normal definition? > a + bi < c + di iff sqrt(a**2 + b**2) < sqrt(c**2, d**2) > > How do you implement a set without sorting? > > Are you expecting better than O(log n)? Of course, hashing does O(1) (most of the time, with a sane hash of course.) Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple calculation error
On Jan 4, 12:30 pm, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Francois Liot wrote: > > > I observed a strange calculation answer, on both python 2.3.4 and 2.4.4 > > >> >> print 753343.44 - 753361.89 > > > -18.450001 > > >> >> print ( (753361.89*100) - (753343.44*100) ) / 100 > > > 18.45 > > > Can somebody help me to play correctly with decimal values? > If the OP is questioning the change in sign, the operands are reversed in the second statement. -- Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple calculation error
Francois Liot wrote: > > I observed a strange calculation answer, on both python 2.3.4 and 2.4.4 > >> >> print 753343.44 - 753361.89 > > -18.450001 > >> >> print ( (753361.89*100) - (753343.44*100) ) / 100 > > 18.45 > > Can somebody help me to play correctly with decimal values? A 64-bit binary floating point number can hold values between -1e308 and +1e308, in only 64 bits of memory. Since 1e308 is a *lot* larger than float(2**64) = ~1.8e19, it does this by splitting the number in a binary fraction, and a multiplier (stored as an exponent). Unfortunately, very few decimal fractions can be *exactly* represented as binary fractions, so you often get representation errors: http://docs.python.org/tut/node16.html This is usually not much of a problem, since you usually end up rounding things to a suitable number of decimals or significant digits when you print them anyway (see below), but repr() doesn't do that -- it always outputs enough digits to get back the *exact* binary representation if you convert the string back to a floating point value again. In practice, you can usually ignore this; just use the appropriate output methods, and things will just work: While pathological cases do exist, for most casual use of floating-point arithmetic you'll see the result you expect in the end if you simply round the display of your final results to the number of decimal digits you expect. str() usually suffices, and for finer control see the discussion of Python's % format operator: the %g, %f and %e format codes supply flexible and easy ways to round float results for display. (from the above link) If you really need full control over this, no matter what, use the Decimal type, as provided by the decimal module in the standard library. See the library reference for the full story. -- http://mail.python.org/mailman/listinfo/python-list
RE: Simple calculation error
No the change of sign is due to a fake copy and past, My question was related to decimal calculation. Thanks, Francois Liot -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paul McGuire Sent: Friday, January 04, 2008 1:46 PM To: python-list@python.org Subject: Re: Simple calculation error On Jan 4, 12:30 pm, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Francois Liot wrote: > > > I observed a strange calculation answer, on both python 2.3.4 and 2.4.4 > > >> >> print 753343.44 - 753361.89 > > > -18.450001 > > >> >> print ( (753361.89*100) - (753343.44*100) ) / 100 > > > 18.45 > > > Can somebody help me to play correctly with decimal values? > If the OP is questioning the change in sign, the operands are reversed in the second statement. -- Paul -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
fastest method to choose a random element
Hello, This is a question for the best method (in terms of performance only) to choose a random element from a list among those that satisfy a certain property. This is the setting: I need to pick from a list a random element that satisfies a given property. All or none of the elements may have the property. Most of the time, many of the elements will satisfy the property, and the property is a bit expensive to evaluate. Chance of having the property are uniform among elements. A simple approach is: import random def random_pick(a_list,property): '''Returns a random element from a list that has the property Returns None if no element has the property ''' random.shuffle(a_list) for i in a_list: if property(i): return i but that requires to shuffle the list every time. A second approach, that works if we know that at least one element of the list has the property, is: import random def random_pick(a_list,property): '''Returns a random element from a list that has the property Loops forever if no element has the property ''' while 1: i=random.choice(a_list) if property(i): return i which is more efficient (on average) if many elements of the list have the property and less efficient if only few elements of the list has the property (and goes crazy if no element has the property) Yet another one: import random def random_pick(a_list,property): '''Returns a random element from a list that has the property ''' b_list=[x for x in a_list if property(x)] try: return random.choice(b_list) finally: return None but this one checks the property on all the elements, which is no good. I don't need strong random numbers, so a simple solution like: import random globalRNG=random.Random() def random_pick(a_list,property): '''Returns a random element from a list that has the property Works only if len(a_list)+1 is prime: uses Fermat's little theorem ''' a=globalRNG(1,len(a_list)) ind=a for i in xrange(len(a_list)): x=a_list[a-1] if property(x):return x ind*=a but this works only if len(a_list)+1 is prime!!! Now this one could be saved if there were an efficient method to find a prime number given than a number n but not very much greater... Any other ideas? Thanks everybody -- http://mail.python.org/mailman/listinfo/python-list
MRO Error on Multiple Inheritance?
I'm working through Wesley Chun's CPP2e and got this error on 13.11.1, pp 548 where his interpreter snippet shows no problems: ActivePython 2.5.1.1 (ActiveState Software Inc.) b Python 2.5.1 (r251:54863, May 1 2007, 17:47:05) [ win32 Type "help", "copyright", "credits" or "license" f >>> class A(object): pass ... >>> class B(A): pass ... >>> class C(B): pass ... >>> class D(A, B): pass ... Traceback (most recent call last): File "", line 1, in TypeError: Error when calling the metaclass bases Cannot create a consistent method resolution order (MRO) for bases A, B (I submitted the problem to the author but I'm not sure I'll ever hear back.) I'm guessing that this kind of diamond inheritance is prohibited by the interpreter, and that his lack of error messages from the interpretation is due to actually leaving out the "class B(A): pass" Can someone shed light? Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Tabnanny errors when Migrating Python 2.4 code to 2.5
On Jan 5, 3:56 am, [EMAIL PROTECTED] wrote: > Hi, > > When Python 2.5 first came out, I eagerly downloaded it and > immediately had issues with getting it to run my 2.4 code. So I just > stuck to 2.4. However, I decided this week that I really should try to > get 2.5 to work. Does anyone know why code that works perfectly for > months in a 2.4 environment throws indentation errors in 2.5? No, not until you go to the bother of reproducing the problem with a small file, tell us what platform you are on, how you are running this code (IDLE, shell prompt, ...), how you installed Python 2.5 (2.5.1?), ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Details about pythons set implementation
> bukzor <[EMAIL PROTECTED]> (B) wrote: >B> Why cant you implement < for complex numbers? Maybe I'm being naive, >B> but isn't this the normal definition? >B> a + bi < c + di iff sqrt(a**2 + b**2) < sqrt(c**2, d**2) There doesn't exist a `normal' definition of < for the complex numbers. For example you would expect that x URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: Question on os.tempnam() vulnerability
[EMAIL PROTECTED] wrote: > Does any one know what kind of security risk these message are > suggesting? > f = os.tempnam() > __main__:1: RuntimeWarning: tempnam is a potential security risk to > your program f > '/tmp/filed4cJNX' > g = os.tmpnam() > __main__:1: RuntimeWarning: tmpnam is a potential security risk to > your program g > '/tmp/fileENAuNw' you get a name instead of a file, so someone else can create that file after you've called tempnam/tmpnam, but before you've actually gotten around to create the file yourself. which means that anyone on the machine might be able to mess with your application's data. use the functions marked as "safe" in the tempfile module instead. -- http://mail.python.org/mailman/listinfo/python-list
Question on os.tempnam() vulnerability
Hello, Does any one know what kind of security risk these message are suggesting? >>> f = os.tempnam() __main__:1: RuntimeWarning: tempnam is a potential security risk to your program >>> f '/tmp/filed4cJNX' >>> g = os.tmpnam() __main__:1: RuntimeWarning: tmpnam is a potential security risk to your program >>> g '/tmp/fileENAuNw' Thanks, ~cw -- http://mail.python.org/mailman/listinfo/python-list
Re: MRO Error on Multiple Inheritance?
Ming wrote: > TypeError: Error when calling the metaclass bases > Cannot create a consistent method resolution > order (MRO) for bases A, B > > (I submitted the problem to the author but I'm not sure I'll ever hear > back.) I'm guessing that this kind of diamond inheritance is > prohibited by the interpreter, and that his lack of error messages > from the interpretation is due to actually leaving out the "class > B(A): pass" or, alternatively, leaving out the (object) in the first class definition: >>> class A: pass ... >>> class B(A): pass ... >>> class D(A, B): pass ... >>> -- http://mail.python.org/mailman/listinfo/python-list
Re: fastest method to choose a random element
On Jan 4, 2008 2:55 PM, <[EMAIL PROTECTED]> wrote: > Hello, > This is a question for the best method (in terms of performance > only) to choose a random element from a list among those that satisfy > a certain property. I would automatically use random.choice(filter(pred_func, a_list)). You just have to catch the possible IndexError. This is the setting: I need to pick from a list a random element > that satisfies a given property. All or none of the elements may have > the property. Most of the time, many of the elements will satisfy the > property, and the property is a bit expensive to evaluate. Chance of > having the property are uniform among elements. > > A simple approach is: > > import random > def random_pick(a_list,property): >'''Returns a random element from a list that has the property > >Returns None if no element has the property >''' >random.shuffle(a_list) >for i in a_list: >if property(i): return i I'm pretty sure you don't want to use a destructive random_pick function. You'll have to shuffle a copy instead to avoid that problem. > > but that requires to shuffle the list every time. > > A second approach, that works if we know that at least one element of > the list has the property, is: > > import random > def random_pick(a_list,property): >'''Returns a random element from a list that has the property > >Loops forever if no element has the property >''' >while 1: >i=random.choice(a_list) >if property(i): return i > > which is more efficient (on average) if many elements of the list have > the property and less efficient if only few elements of the list has > the property (and goes crazy if no element has the property) That's awful. Here's another linear time idea, returning the nearest element that satisfies the predicate. offset = random.randrange(len(a_list)) for n in xrange(len(a_list)): ix = (offset + n) % len(a_list) if predicate(a_list[ix]): return a_list[ix] raise ValueError('no element has the property') The possible problem is that large strings of elements in a row that don't match the predicate greatly increase the odds of getting the following element that *does* match the predicate. Worst case is two predicate matched elements in a row, surrounded by a bunch of non-matched elements. > Yet another one: > > import random > def random_pick(a_list,property): >'''Returns a random element from a list that has the property >''' >b_list=[x for x in a_list if property(x)] >try: >return random.choice(b_list) >finally: return None > > but this one checks the property on all the elements, which is no > good. Is it really expensive to check the property? That would mitigate against the filter solution and for the other one I posted. This seems to be a case of trying to solve a data problem functionally. It'd be better to store your data differently if this will be a frequent operation and you simply can't afford to call the predicate on all the elements. Incidentally, try not to shadow builtin names like 'property'. -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: fastest method to choose a random element
On Jan 4, 2008 3:47 PM, Neil Cerutti <[EMAIL PROTECTED]> wrote: > On Jan 4, 2008 2:55 PM, <[EMAIL PROTECTED]> wrote: > > > Hello, > > This is a question for the best method (in terms of performance > > only) to choose a random element from a list among those that satisfy > > a certain property. > > > > A simple approach is: > > > > import random > > def random_pick(a_list,property): > >'''Returns a random element from a list that has the property > > > >Returns None if no element has the property > >''' > >random.shuffle(a_list) > >for i in a_list: > >if property(i): return i > > > I'm pretty sure you don't want to use a destructive random_pick function. > You'll have to shuffle a copy instead to avoid that problem. > I thought of another one based on combining the above with the linear search idea, minimizing the calls to the predicate function. indexes = range(len(a_list)) random.shuffle(indexes) for ix in indexes: if predicate(a_list[ix]) return a_list[ix] raise ValueError('no matching element in list') -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: MRO Error on Multiple Inheritance?
On Jan 4, 2008 3:03 PM, Ming <[EMAIL PROTECTED]> wrote: > I'm working through Wesley Chun's CPP2e and got this error on 13.11.1, > pp 548 where his interpreter snippet shows no problems: > > ActivePython 2.5.1.1 (ActiveState Software Inc.) b > Python 2.5.1 (r251:54863, May 1 2007, 17:47:05) [ > win32 > Type "help", "copyright", "credits" or "license" f > >>> class A(object): pass > ... > >>> class B(A): pass > ... > >>> class C(B): pass > ... > >>> class D(A, B): pass > ... > Traceback (most recent call last): > File "", line 1, in > TypeError: Error when calling the metaclass bases >Cannot create a consistent method resolution > order (MRO) for bases A, B The mro of new-style classes changed between Python 2.2 and 2.3. Perhaps Mr. Chun's code was written for 2.2. See http://www.python.org/download/releases/2.3/mro/ -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: Tabnanny errors when Migrating Python 2.4 code to 2.5
On Jan 4, 2:06 pm, John Machin <[EMAIL PROTECTED]> wrote: > On Jan 5, 3:56 am, [EMAIL PROTECTED] wrote: > > > Hi, > > > When Python 2.5 first came out, I eagerly downloaded it and > > immediately had issues with getting it to run my 2.4 code. So I just > > stuck to 2.4. However, I decided this week that I really should try to > > get 2.5 to work. Does anyone know why code that works perfectly for > > months in a 2.4 environment throws indentation errors in 2.5? > > No, not until you go to the bother of reproducing the problem with a > small file, tell us what platform you are on, how you are running this > code (IDLE, shell prompt, ...), how you installed Python 2.5 > (2.5.1?), ... I'm using Windows XP, using IDLE (which was mentioned already) and I downloaded the 2.5.1 exe/msi file from python.org to install it. I have yet to find a simple one which exhibits the issue to post. It seems to happen to my complex files, not the simple ones. Sorry to bother you. Mike -- http://mail.python.org/mailman/listinfo/python-list