Re: Is this a safe use of eval?
Frank Millman wrote: > Hi all > > I know that the use of 'eval' is discouraged because of the dangers of > executing untrusted code. > > Here is a variation that seems safe to me, but I could be missing > something. > > I have a class, and the class has one or more methods which accept various > arguments and return a result. > > I want to accept a method name and arguments in string form, and 'eval' it > to get the result. > > Assume I have an instance called my_inst, and a method called 'calc_area', > with arguments w and h. > > I then receive my_string = 'calc_area(100, 200)'. > result = eval('my_inst.{0}'.format(my_string)) > > This will only work if the string contains a valid method name with valid > arguments. > > Can anyone see anything wrong with this? How do you prevent that a malicious source sends you my_string = 'calc_area(__import__("os").system("rm important_file") or 100, 200)' instead? -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a safe use of eval?
Thanks, Paul and Peter. It seemed like a good idea at the time. Thank you for straightening me out. Frank -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a safe use of eval?
On Thu, 2011-02-24 at 10:48 +0200, Frank Millman wrote: > Hi all > > I know that the use of 'eval' is discouraged because of the dangers of > executing untrusted code. > > Here is a variation that seems safe to me, but I could be missing something. > > I have a class, and the class has one or more methods which accept various > arguments and return a result. > > I want to accept a method name and arguments in string form, and 'eval' it > to get the result. > > Assume I have an instance called my_inst, and a method called 'calc_area', > with arguments w and h. > > I then receive my_string = 'calc_area(100, 200)'. > > >>> result = eval('my_inst.{0}'.format(my_string)) > > This will only work if the string contains a valid method name with valid > arguments. > > Can anyone see anything wrong with this? Yes. A serious problem. Here's an example of what you describe: >>> class MyClass(object): ... def calc_area(self,x,y): ... return 42 ... >>> inst = MyClass() >>> >>> def testit(query): ... return eval("inst.{0}".format(query)) >>> It works as you expect when used properly: >>> testit('calc_area(3,4)') 42 But actually it allows all sorts of nasty things. Watch me open an arbitrary file on your system (assuming you have permission of course). First find out how to access the "file" builtin: >>> testit('__class__.__mro__[-1].__subclasses__().index(file)') 58 Great, plug that in and we're in business: >>> testit('__class__.__mro__[-1].__subclasses__()[58]("/secret/file","w")') Traceback (most recent call last): File "", line 1, in File "", line 2, in testit File "", line 1, in IOError: [Errno 2] No such file or directory: '/secret/file' >>> So please, don't do this! :-) Ryan -- Ryan Kelly http://www.rfk.id.au | This message is digitally signed. Please visit r...@rfk.id.au| http://www.rfk.id.au/ramblings/gpg/ for details signature.asc Description: This is a digitally signed message part -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a safe use of eval?
On Thu, 2011-02-24 at 20:13 +1100, Ryan Kelly wrote: > On Thu, 2011-02-24 at 10:48 +0200, Frank Millman wrote: > > Hi all > > > > I know that the use of 'eval' is discouraged because of the dangers of > > executing untrusted code. > > > > Here is a variation that seems safe to me, but I could be missing something. > > > > I have a class, and the class has one or more methods which accept various > > arguments and return a result. > > > > I want to accept a method name and arguments in string form, and 'eval' it > > to get the result. > > > > Assume I have an instance called my_inst, and a method called 'calc_area', > > with arguments w and h. > > > > I then receive my_string = 'calc_area(100, 200)'. > > > > >>> result = eval('my_inst.{0}'.format(my_string)) > > > > This will only work if the string contains a valid method name with valid > > arguments. > > > > Can anyone see anything wrong with this? > > Yes. A serious problem. > > > But actually it allows all sorts of nasty things. Watch me open an > arbitrary file on your system (assuming you have permission of course). > > >>> > testit('__class__.__mro__[-1].__subclasses__()[58]("/secret/file","w")') > Traceback (most recent call last): > File "", line 1, in > File "", line 2, in testit > File "", line 1, in > IOError: [Errno 2] No such file or directory: '/secret/file' > >>> Just to elaborate a little more. Once you've got a reference to 'object' the game it pretty much over - you can walk its subclasses to get to all kinds of nasty things like 'file'. Since this was a newstyle class, getting to 'object' was easy - it's always the class's final base class (i.e. __mro__[-1]). You might think that using an old-style class would save you: >>> class MyClass: ... pass ... >>> inst = MyClass() >>> inst.__class__.__mro__[-1] Traceback (most recent call last): File "", line 1, in AttributeError: class MyClass has no attribute '__mro__' But there will almost always be a reference to a builtin type hanging around somewhere. Builtin types are all newstyle classes, and hence have 'object' in their mro. For example: >>> inst.__dict__.__class__.__mro__[-1] >>> Or perhaps: >>> inst.__class__.__name__.__class__.__mro__[-1] It's pretty much impossible to prevent this kind of thing in python. You might get away with it if you parse the string looking for suspicious characters e.g. dots. But that will limit your grammar and I won't be surprised if there's still a way around it. > So please, don't do this! :-) So what's the alternative? As suggested by another poster, for simple cases use the ast.literal_eval function. For anything more complicated, use PyParsing to generate your own mini language and interpret it yourself. It's really pretty simple once you get in the right head-space. Try some of the links from this SO post if you want to start down this path: http://stackoverflow.com/questions/1545403/math-expression-evaluation Cheers, Ryan -- Ryan Kelly http://www.rfk.id.au | This message is digitally signed. Please visit r...@rfk.id.au| http://www.rfk.id.au/ramblings/gpg/ for details signature.asc Description: This is a digitally signed message part -- http://mail.python.org/mailman/listinfo/python-list
Re: Struggling to convert a mysql datetime object to a python string of a different format
try this test = time.time(2011, 2, 1, 2, 4, 10) # this is your datetime object from mysql print time.mktime(test.timetuple()) hope this would help you > On Wednesday, August 04, 2010 7:40 PM ? wrote: > Okey, i have many hours now struggling to convert a mysql datetime > field that i retreive to a string of this format '%d %b, %H:%M' > > I google a lot but couldnt found out how to format it being a string > > Here si the code so far: > > try: > cursor.execute( ''' SELECT host, hits, date FROM visitors WHERE page > =3D '%s' ORDER BY date DESC ''' % (page) ) > except MySQLdb.Error: > print( "Error %d: %s" % (e.args[0], e.args[1]) ) > else: > print ( ''' ( =C5=F0=E9=F3=EA=DD=F0=F4=E7=F2 ) - ( = > =C5=F0=E9=F3=EA=DD=F8=E5=E9=F2 ) > - ( =C7=EC=E5=F1=EF=EC=E7=ED=DF=E1 ) ''' ) > print ( '' ) > > results =3D cursor.fetchall() > > for row in results: > print ( ''' ''' ) > > for entry in row: > entry =3D datetime.datetime.strftime( entry, '%d %b, %H:%M' ) #!!! > this is wrong! > print ( ''' %s ''' % entry ) > > sys.exit(0) > > Apart from that i do not know how iam supposed to print it, because the > date string is the 3rd string in every row of the dataset. > > Please help me out! >> On Thursday, August 05, 2010 4:55 AM Dennis Lee Bieber wrote: >> gmane.comp.python.general: >> >> As you state, it is the third item in each returned row... So why >> are you trying to treat EVERY item in the row as a date? >> >> Since MySQLdb appears to return datetime objects (my quick test is >> showing datetime.date for dates in a test database) you should be >> probably be using >> >> formatted_entry = entry.strftime("%d... %M") >> >> to do the formatting as string >> >> >> >> -- >> Wulfraed Dennis Lee Bieber AF6VN >> wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/ >>> On Thursday, August 05, 2010 12:31 PM ? wrote: >>> rote: >>> r entry in row: >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0entry =3D datetime.datetime.strftime( entry, '%d= >>> %b, %H:%M' ) #!!! >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0print ( ''' %s ''' % entry ) >>> turned row... So why >>> >>> Because when i try to prin the 3 items liek that >>> >>> print row[0], row[1], row[2] >>> >>> it gives me an error, so i dont knwo how to tell it how to print the >>> 3rd item differently. >>> >>> >>> >>> cts (my quick test is >>> ) >>> >>> I tried that myself yesterday but look it fails to the following >>> message >>> >>> /home/webville/public_html/cgi-bin/index.py >>> 63 >>> 64 for entry in row: >>> 65 formatted_entry =3D >>> entry.strftime('%d %b, %H:%M') >>> 66 print ( ''' %s ''' % >>> entry ) >>> 67 >>> formatted_entry undefined, entry =3D '178-124-186.dynamic.cyta.gr', >>> entry.strftime undefined >>> AttributeError: 'str' object has no attribute 'strftime' >>> args =3D ("'str' object has no attribute 'strftime'",) On Thursday, August 05, 2010 2:52 PM ? wrote: Hey i made it! :-) dataset = cursor.fetchall() for row in dataset: print ( ''' ''' ) date = row[2].strftime( '%d %b, %H:%M' ) print ( ''' %s%s %s ''' % ( row[0], row[1], date ) ) Unfortunately had to ditch the 'for entry in row' line because could not iterate over the items of the row. Could you please shoe me how could i do the same thing with iteration?! Thanks! > On Thursday, August 05, 2010 3:09 PM Tim Chase wrote: > On 08/05/10 13:52, ?? wrote: > > Well, depending on whether "row" is a tuple or a list, you can do > either > > row[2] = row[2].strftime(...) # if it is a list > > or you can just iterate over a predefined list/tuple: > > for row in dataset: > print ("") > for item in (row[0], row[1], row[2].strftime(...)): > print ("%s print ("") > > Though I think I'd make it a bit clearer by naming the fields: > > for host, hits, dt in dataset: > print ("") > for item in (host, hits, dt.strftime(...)): > print ("%s" % item) > print ("") > > Or perhaps even just > > print ("".join("%s" % item > for item in (host, hits, dt.strftime(...)) > ) > > Whichever you prefer. I think I am partial to the 2nd-from-last > version, especially as the list of fields may grow. > > -tkc >> On Thursday, August 05, 2010 5:01 PM ? wrote: >> As i have it the returned 'dataset' is stored line per line to 'row'. >> >> So, >> 'dataset' in here is a 'list of tuples' right? >> and >> 'row' in here is a tuple form the above list of tuples right? >> >> Am i understanding this correctly?! >> >> >> >> It was a tuple. But it migth as well be a list too?!?! >> >> Could 'dataset' be a 'list of lists' as well? >> >> How one would know
Re: Struggling to convert a mysql datetime object to a python string of a different format
On 02/24/2011 04:34 AM, rahul mishra wrote: > try this > > test = time.time(2011, 2, 1, 2, 4, 10) > # this is your datetime object from mysql > > print time.mktime(test.timetuple()) > > hope this would help you > > You do realize that email was sent over four months ago, correct? See: >> On Wednesday, August 04, 2010 7:40 PM ? wrote: -- Corey Richardson -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
On Feb 23, 5:22 pm, grobs456 wrote: > Hi, > > I am trying to work through the tutorial at:http://docs.python.org/tutorial/ > > The issue I am facing is with regards to the discussion about > "Invoking the Interpreter" and "Executable Python Scripts". It was > rather hazy in my opinion. > > see:http://docs.python.org/tutorial/interpreter.html > > I realize I can double click on a .py file and Windows treats it as an > executable but the command prompt pops in and out so quickly that I > can't see the results of my script. > > I would prefer to follow along with the tutorial, compiling new ticks > and topics into a growing .py file, that I error check of course(in > case the same variables are used throughout the tutorial), and then > run that file to see the results as if I would have typed each line in > one at a time. This way I have something usable to refer back to for > learning purposes and for future coding projects instead of having to > rely on the tutorial itself for my reference moving forward. Any > ideas? > > Python is installed at: > C:\Python27 > > and I ran: > set path=%path%;C:\python27 > > #do i have to run the above each time I open up a session? > #Also, note that I do not see the any system environment variables > get updated when I run the above. Greetings, I too have been learning Python on Windows in recent months. I have found using IDLE more than adequate for my purposes. When you right- click on a .py file "Edit with IDLE" is one of the options (if you have a standard Python install). Selecting "run" from IDLE opens up a Python shell and runs the script. Alternatitively, you can open IDLE directly then load your script into it. Unless you need to directly enter Windows commands at a DOS prompt, running scripts from IDLE's Python shell makes for a superior command line for Python, since it is syntax aware (e.g. helps with code indentation) and allows for easy cut-and-paste of previously typed commands. IDLE is a bit clunky compared to commerical IDEs, but is excellent for doing quick experiments. -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes inheritance issue
On Feb 23, 9:38 am, Steve wrote: > After looking at some metaclass examples it appears this information > is readily available. A metaclass gets a dictionary containing > information about the parent class (or should, at least). What examples did you look at? > It seems > like it must have this information in order to dynamically make > decisions about how to create classes... So, "bug" or not, shouldn't > this just work? No. Information about parent class members is available if you dig for it but it doesn't "just work". A metaclass gets three pieces of information it uses when constructing a class: the name of the class, a list of bases, and a dictionary containing everything defined in the class's scope (and only the class's scope, not the scope of any base classes). Some, if not most, metaclasses inspect and modify this dictionary before passing it to the type constructor (type.__new__); inheritance hasn't even come into play at that point. A metaclass can look at the list of bases and try to extract attributes from them, but that's not just working; that's digging. (Needless to say, a lot of implementors don't go through the effort to dig.) > Is there something that prevents it from being > implemented? Would this break something? As I said, it's inherently a chicken-and-egg problem. You have a situation where you want to inherit the information needed to create a class, but inheritance doesn't come into play until the class is created. I guess you could elimiate the paradox by breaking down type construction into steps (set up the inheritance relationships first, then construct the type object, giving the metaclass time to get data from the bases). Some other language will have to try that, though. Yes it would break things. Not a lot of things but there cases where you don't want to inherit. I use the following pattern fairly often: class KeepTrackOfSubtypesMetaclass(type): subtypes = {} def __new__(metatype,name,bases,class_dct): key = class_dct.get('key') self = type.__new__(metatype,name,bases,class_dct) if key is not None: metatype.subtypes[key] = self return self Any instance of this metaclass that defines key in its scope will be added to the dict of subtypes. But I don't want a derived class to overwrite its parent's entry in the subtype dict--it should define its own key. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Unload a DLL
Hi Ppl, I'm loading a dll using the *cdll.LoadLibrary *function. How can I release the dll after I'm done with it. Are there any functions to do this. Thanks, Sathish -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a safe use of eval?
Am 24.02.2011 10:01, schrieb Peter Otten: > How do you prevent that a malicious source sends you > > my_string = 'calc_area(__import__("os").system("rm important_file") or 100, > 200)' > > instead? By using something like http://code.activestate.com/recipes/496746-restricted-safe-eval/ . With a combination of AST inspection and restricted builtins you can create a restricted eval function that e.g. doesn't allow function calls, raising or excepting exceptions and prevents access to members with a leading _. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote: > The IEEE 754 compliant FPU on most machines today, though, has an 80-bit > internal representation. If you do a sequence of operations that retain > all the intermediate results in the FPU registers, you get 16 more bits > of precision than if you store after each operation. That's a big if though. Which languages support such a thing? C doubles are 64 bit, same as Python. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: 2to3 chokes on bad character
On Feb 23, 7:47 pm, "Frank Millman" wrote: > Hi all > > I don't know if this counts as a bug in 2to3.py, but when I ran it on my > program directory it crashed, with a traceback but without any indication of > which file caused the problem. > [traceback snipped] > UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 5055: > invalid start byte > > On investigation, I found some funny characters in docstrings that I > copy/pasted from a pdf file. > > Here are the details if they are of any use. Oddly, I found two instances > where characters 'look like' apostrophes when viewed in my text editor, but > one of them was accepted by 2to3 and the other caused the crash. > > The one that was accepted consists of three bytes - 226, 128, 153 (as > reported by python 2.6) How did you incite it to report like that? Just use repr(the_3_bytes). It'll show up as '\xe2\x80\x99'. >>> from unicodedata import name as ucname >>> ''.join(chr(i) for i in (226, 128, 153)).decode('utf8') u'\u2019' >>> ucname(_) 'RIGHT SINGLE QUOTATION MARK' What you have there is the UTF-8 representation of U+2019 RIGHT SINGLE QUOTATION MARK. That's OK. or 226, 8364, 8482 (as reported by python3.2). Sorry, but you have instructed Python 3.2 to commit a nonsense: >>> [ord(chr(i).decode('cp1252')) for i in (226, 128, 153)] [226, 8364, 8482] In other words, you have taken that 3-byte sequence, decoded each byte separately using cp1252 (aka "the usual suspect") into a meaningless Unicode character and printed its ordinal. In Python 3, don't use repr(); it has undergone the MHTP transformation and become ascii(). > > The one that crashed consists of a single byte - 146 (python 2.6) or 8217 > (python 3.2). >>> chr(146).decode('cp1252') u'\u2019' >>> hex(8217) '0x2019' > The issue is not that 2to3 should handle this correctly, but that it should > give a more informative error message to the unsuspecting user. Your Python 2.x code should be TESTED before you poke 2to3 at it. In this case just trying to run or import the offending code file would have given an informative syntax error (you have declared the .py file to be encoded in UTF-8 but it's not). > BTW I have always waited for 'final releases' before upgrading in the past, > but this makes me realise the importance of checking out the beta versions - > I will do so in future. I'm willing to bet that the same would happen with Python 3.1, if a 3.1 to 3.2 upgrade is what you are talking about -- http://mail.python.org/mailman/listinfo/python-list
Why this difference?
file my.txt: === 0 beb 1 qwe 2 asd 3 hyu 4 zed 5 asd 6 oth = py script: === import sys sys.stdin = open('88.txt', 'r') t = sys.stdin.readlines() t = map(lambda rec: rec.split(), t) print t print t[2][1] == t[5][1] print t[2][1] is t[5][1] print '==' a = 'asd' b = 'asd' print a is b output: === [['0', 'beb'], ['1', 'qwe'], ['2', 'asd'], ['3', 'hyu'], ['4', 'zed'], ['5', 'as d'], ['6', 'oth']] True False == True -- http://mail.python.org/mailman/listinfo/python-list
Re: pattern matching
On Feb 23, 9:11 pm, monkeys paw wrote: > if I have a string such as '01/12/2011' and i want > to reformat it as '20110112', how do i pull out the components > of the string and reformat them into a DDMM format? > > I have: > > import re > > test = re.compile('\d\d\/') > f = open('test.html') # This file contains the html dates > for line in f: > if test.search(line): > # I need to pull the date components here What you need are parentheses, which capture part of the text you're matching. Each set of parentheses creates a "group". To get to these groups, you need the match object which is returned by re.search. Group 0 is the entire match, group 1 is the contents of the first set of parentheses, and so forth. If the regex does not match, then re.search returns None. DATA FILE (test.html): David02/19/1967 Susan05/23/1948 Clare09/22/1952 BP08/27/1990 Roger12/19/1954 CODE: import re rx_test = re.compile(r'(\d{2})/(\d{2})/(\d{4})') f = open('test.html') for line in f: m = rx_test.search(line) if m: new_date = m.group(3) + m.group(1) + m.group(2) print "raw text: ",m.group(0) print "new date: ",new_date print OUTPUT: raw text: 02/19/1967 new date: 19670219 raw text: 05/23/1948 new date: 19480523 raw text: 09/22/1952 new date: 19520922 raw text: 08/27/1990 new date: 19900827 raw text: 12/19/1954 new date: 19541219 -- http://mail.python.org/mailman/listinfo/python-list
Re: Why this difference?
The 1st "False" is not surprising for me. It's the 2nd "True" is a bit hmmm... ok, it doesn't matter == Have a nice day! -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml
On 24-Feb-11 03:20 AM, Stefan Behnel wrote: MRAB, 24.02.2011 01:25: On 24/02/2011 00:10, Colin J. Williams wrote: Could someone please let me know whether lxml is available for Windows XP?. If so, is it available for Python 2.7? The latest stable release is here: http://pypi.python.org/pypi/lxml/2.2.8 Not quite the latest "stable release" (that would be 2.3), but at least one that's pre-built for Windows. Stefan Thanks to both respondents. I had tried easy_install before. It now looks clean and importable. Colin W. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
Steven D'Aprano wrote: On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote: The IEEE 754 compliant FPU on most machines today, though, has an 80-bit internal representation. If you do a sequence of operations that retain all the intermediate results in the FPU registers, you get 16 more bits of precision than if you store after each operation. That's a big if though. Which languages support such a thing? C doubles are 64 bit, same as Python. Assembly! :) ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why this difference?
Den 24.02.11 13.41, skrev n00m: The 1st "False" is not surprising for me. It's the 2nd "True" is a bit hmmm... ok, it doesn't matter == Have a nice day! I am no expert, but I think python re-uses some integer and string objects. For instance, if you create the object int(2) it may be re-used later if you have several 2 objects in your code. This is to save some memory, or some other performance hack. Don't rely on it. For instance: >>> a = 100 >>> b = 100 >>> a is b True >>> a = 2**60 >>> b = 2**60 >>> a is b False Strange, but it's just like this! Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: 2to3 chokes on bad character
John Machin wrote: > On Feb 23, 7:47 pm, "Frank Millman" wrote: >> Hi all >> >> I don't know if this counts as a bug in 2to3.py, but when I ran it on my >> program directory it crashed, with a traceback but without any indication >> of which file caused the problem. >> > [traceback snipped] > >> UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 5055: >> invalid start byte >> >> On investigation, I found some funny characters in docstrings that I >> copy/pasted from a pdf file. >> >> Here are the details if they are of any use. Oddly, I found two instances >> where characters 'look like' apostrophes when viewed in my text editor, >> but one of them was accepted by 2to3 and the other caused the crash. >> >> The one that was accepted consists of three bytes - 226, 128, 153 (as >> reported by python 2.6) > > How did you incite it to report like that? Just use repr(the_3_bytes). > It'll show up as '\xe2\x80\x99'. > > >>> from unicodedata import name as ucname > >>> ''.join(chr(i) for i in (226, 128, 153)).decode('utf8') > u'\u2019' > >>> ucname(_) > 'RIGHT SINGLE QUOTATION MARK' > > What you have there is the UTF-8 representation of U+2019 RIGHT SINGLE > QUOTATION MARK. That's OK. > > or 226, 8364, 8482 (as reported by python3.2). > > Sorry, but you have instructed Python 3.2 to commit a nonsense: > > >>> [ord(chr(i).decode('cp1252')) for i in (226, 128, 153)] > [226, 8364, 8482] > > In other words, you have taken that 3-byte sequence, decoded each byte > separately using cp1252 (aka "the usual suspect") into a meaningless > Unicode character and printed its ordinal. > > In Python 3, don't use repr(); it has undergone the MHTP > transformation and become ascii(). > >> >> The one that crashed consists of a single byte - 146 (python 2.6) or 8217 >> (python 3.2). > > >>> chr(146).decode('cp1252') > u'\u2019' > >>> hex(8217) > '0x2019' > > >> The issue is not that 2to3 should handle this correctly, but that it >> should give a more informative error message to the unsuspecting user. > > Your Python 2.x code should be TESTED before you poke 2to3 at it. In > this case just trying to run or import the offending code file would > have given an informative syntax error (you have declared the .py file > to be encoded in UTF-8 but it's not). The problem is that Python 2.x accepts arbitrary bytes in string constants. No error message or warning: $ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> with open("tmp.py", "w") as f: # prepare the broken script ... f.write("# -*- coding: utf-8 -*-\nprint 'bogus char: \x92'\n") ... >>> $ cat tmp.py # -*- coding: utf-8 -*- print 'bogus char: �' $ python2.6 tmp.py bogus char: � $ 2to3-3.2 tmp.py [traceback snipped] UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 43: invalid start byte In theory 2to3 could be changed to take the same approach as os.listdir(), but as in the OP's example occurences of the problem are likely to be editing accidents. -- http://mail.python.org/mailman/listinfo/python-list
Re: 2to3 chokes on bad character
"John Machin" wrote: On Feb 23, 7:47 pm, "Frank Millman" wrote: [snip lots of valuable info] The issue is not that 2to3 should handle this correctly, but that it should give a more informative error message to the unsuspecting user. Your Python 2.x code should be TESTED before you poke 2to3 at it. In this case just trying to run or import the offending code file would have given an informative syntax error (you have declared the .py file to be encoded in UTF-8 but it's not). Thank you, John - this is the main lesson. The file that caused the error has a .py extension, and looks like a python file, but it just contains documentation. It has never been executed or imported. As you say, if I had tried to run it under Python 2 it would have failed straight away. In these circumstances, it is unreasonable to expect 2to3 to know what to do with it, so it is definitely not a bug. BTW I have always waited for 'final releases' before upgrading in the past, but this makes me realise the importance of checking out the beta versions - I will do so in future. I'm willing to bet that the same would happen with Python 3.1, if a 3.1 to 3.2 upgrade is what you are talking about This is my first look at Python 3, so I am talking about moving from 2.6 to 3.2. In this case, it turns out that it was not a bug, but still, in future I will run some tests when betas are released, just in case I come up with something. Thanks for your response - it was very useful. Frank -- http://mail.python.org/mailman/listinfo/python-list
Re: 2to3 chokes on bad character
"Peter Otten" <__pete...@web.de> wrote John Machin wrote: Your Python 2.x code should be TESTED before you poke 2to3 at it. In this case just trying to run or import the offending code file would have given an informative syntax error (you have declared the .py file to be encoded in UTF-8 but it's not). The problem is that Python 2.x accepts arbitrary bytes in string constants. No error message or warning: Thanks, Peter. I saw this after I replied to John, so this somewhat invalidates my reply. However, John's principle still holds true, and that is the main lesson I have taken away from this. Frank -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a safe use of eval?
"Christian Heimes" wrote Am 24.02.2011 10:01, schrieb Peter Otten: How do you prevent that a malicious source sends you my_string = 'calc_area(__import__("os").system("rm important_file") or 100, 200)' instead? By using something like http://code.activestate.com/recipes/496746-restricted-safe-eval/ . With a combination of AST inspection and restricted builtins you can create a restricted eval function that e.g. doesn't allow function calls, raising or excepting exceptions and prevents access to members with a leading _. Thanks, Christian. I had a look at that recipe, but I must say that Paul's suggestion is much simpler - from ast import literal_eval method_name = 'calc_area' args = literal_eval('(100,200)') result = getattr(my_inst, method_name)(*args) In my case the arguments are all strings or integers, so it looks as if this approach should be safe. Do you see any problem with it? Frank -- http://mail.python.org/mailman/listinfo/python-list
Re: Why this difference?
> Don't rely on it. Hmm I never was about to rely on it. Simply sorta my academic curiosity. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On Thu, 24 Feb 2011 04:56:46 -0800 Ethan Furman wrote: > >> The IEEE 754 compliant FPU on most machines today, though, has an 80-bit > >> internal representation. If you do a sequence of operations that retain > >> all the intermediate results in the FPU registers, you get 16 more bits > >> of precision than if you store after each operation. > > > > That's a big if though. Which languages support such a thing? C doubles > > are 64 bit, same as Python. > > Assembly! :) Really? Why would you need that level of precision just to gather all the students into the auditorium? -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. -- http://mail.python.org/mailman/listinfo/python-list
Today's Hottest actresses
Hello friends. Would you like to know about bollywood hot and beautiful actresses?You can fine a lot of information about bollywood actresses.Just visit www.hotpics00.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: [PySIL] Securing files
Greetings, > The problem isn't so much the database itself, > as I can think of a number of way to encrypt the data it contains, > but some of the data is simply names of image and video files > contained elsewhere in the file-system. Actually, this is something like I had to think through with a prayer/newletter archiving program. > One way that I can think of would be to encode the image/video files > as BLOBS and store them in the database itself, but apart from that > option, can anyone suggest other ways? The problem with this method is that if you need to store large pieces of data there may be a data transmission limit (eg. how long your SQL insert can be) when storing the data. Increasing the size allowed fixes that problem, but writing it straight to the file-system bypasses that problem completely. And if you have a web interface, the question becomes one of a denial of service attacks and sql injection attacks. And if you write in a path accessible by the web server (/home/prayerletters/public_html/...), you've opened the data to the world too, so you must write elsewhere. This, however, leads to your question of people just changing directories and looking at the files themselves. This is where another UserID (prayerletters) comes in. If you can store the files in a directory that only a specific user can access (/home/prayerletters/Letters/...), then the program can run as that UserID to process/display the data. Anyone that can change directories can't get into the directory to view them, unless they are administrators or that specific user. This solves the problem, and is relatively simple compared to encryption. Though, I don't know how simple under windows. This link may be helpful: http://www.python-forum.org/pythonforum/viewtopic.php?f=15&t=19393 It would also complicate installation, as the install program would somehow need to create the new user (if running atand alone). If running in a domain environment, you could use a domain user and path on a server to secure it. As you can see, the working environment really changes how you implement the other UserID. Though, remember to make it not your UserID, because if you change the password, then the program will stop working when you change your password. :) Hope this helps the planning process a little. GPML... -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
D'Arcy J.M. Cain wrote: > On Thu, 24 Feb 2011 04:56:46 -0800 > Ethan Furman wrote: >> > That's a big if though. Which languages support such a thing? C doubles >> > are 64 bit, same as Python. >> >> Assembly! :) > > Really? Why would you need that level of precision just to gather all > the students into the auditorium? You would think so, but darned if some of them don't wind up in a *different* *auditorium*! Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
c:\dev\python>python HelloWorld.py 'python' is not recognized as an internal or external command, operable program or batch file. #I then tried this for a success!: c:\dev\python>c:\python27\python.exe HelloWorld.py Hello WOrld! c:\dev\python>python HelloWorld.py 'python' is not recognized as an internal or external command, operable program or batch file. #I will utilize the advice regarding using IDLE but would like to figure out how to do the above. #Would this help me, per the tutorial?: chmod +x HellowWorld.py #I set my system environment variables to: VARIABLEVALUE PYTHON_HOME c:\python27\python.exe PATH...;%PYTHON_HOME% #after also trying this: VARIABLEVALUE PYTHON_HOME c:\python27 PATH...;%PYTHON_HOME% #ALSO, I was confused by the following: Add into each your *.py script and as the very last line this: raw_input('Press any key to exit...') #Thanks for all the assistance thus far. -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
On Thu, Feb 24, 2011 at 10:34 AM, grobs456 wrote: > c:\dev\python>python HelloWorld.py > 'python' is not recognized as an internal or external command, > operable program or batch file. > > #I then tried this for a success!: > > c:\dev\python>c:\python27\python.exe HelloWorld.py > Hello WOrld! > > c:\dev\python>python HelloWorld.py > 'python' is not recognized as an internal or external command, > operable program or batch file. > > > #I will utilize the advice regarding using IDLE but would like to > figure out how to do the above. > #Would this help me, per the tutorial?: > chmod +x HellowWorld.py > No. That's for POSIX systems (Mac OS X, Unix, Linux) where you want to execute the script just by typing ./HelloWorld.py rather than python HelloWorld.py. The chmod tells the system that you're allowed to execute that file directly. > #I set my system environment variables to: > > VARIABLE VALUE > PYTHON_HOME c:\python27\python.exe > PATH ...;%PYTHON_HOME% > > #after also trying this: > VARIABLE VALUE > PYTHON_HOME c:\python27 > PATH ...;%PYTHON_HOME% > Apparently your PATH isn't getting set properly. If C:\Python27 is on the path, it should find Python just fine. It works on my machine. > #ALSO, I was confused by the following: > Add into each your *.py script and as the very last line this: > raw_input('Press any key to exit...') > > #Thanks for all the assistance thus far. > This is for when you try to execute a script by double-clicking on the file. Every Windows console application will spawn a command prompt when they are run (assuming they aren't run from the command prompt). If they are launched this way, that command prompt will close immediately when the program finishes. By putting a raw_input at the end, you make the program wait for you to enter something before it finishes, so the terminal window will stay open and you can read your program's output. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unload a DLL
On 01/-10/-28163 02:59 PM, Sathish S wrote: Hi Ppl, I'm loading a dll using the *cdll.LoadLibrary *function. How can I release the dll after I'm done with it. Are there any functions to do this. Thanks, Sathish In Windows, the FreeLibrary() call will decrement the load count, and unload the DLL if it reaches zero. it's argument is the handle that LoadLibrary() returned. I have no idea if that's available in the cdll Python interface, but it would make sense. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Python C Extensions
Hi, I have a memory leak problem with my "C" extension module. My C module returns large dictionaries to python, and the dictionaries never get deleted, so the memory for my program keeps growing. I do not know how to delete the dictionary object after it becomes irrelevant. I do not know if the version of python is relevant, I'm using the 2.5 ! Here is the "C" code: PyObject *GetDictionary(PyObject *self, PyObject *args) { PyObject *dict = PyDict_New(); PyObject *key; PyObject *value; char name[128]; for(int i = 0; i < 60; i++) { sprintf(name,"v%d",i); float number = 1.0 * 0.5*i; PyDict_SetItem(dict,Py_BuildValue("s",name),Py_BuildValue("f",number)); } return dict; } And here is the Code that I use in a loop, which causes the program memory to grow: import libpyTestModule as pyTEST bankTEST = {} for j in range(1,10): for k in range(1,10): bankTEST = pyTEST.GetDictionary() del bankTEST Any help will be appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: pattern matching
On Feb 24, 2:11 am, monkeys paw wrote: > if I have a string such as '01/12/2011' and i want > to reformat it as '20110112', how do i pull out the components > of the string and reformat them into a DDMM format? > > I have: > > import re > > test = re.compile('\d\d\/') > f = open('test.html') # This file contains the html dates > for line in f: > if test.search(line): > # I need to pull the date components here I second using an html parser to extact the content of the TD's, but I would also go one step further reformatting and do something such as: >>> from time import strptime, strftime >>> d = '01/12/2011' >>> strftime('%Y%m%d', strptime(d, '%m/%d/%Y')) '20110112' That way you get some validation about the data, ie, if you get '13/12/2011' you've probably got mixed data formats. hth Jon. -- http://mail.python.org/mailman/listinfo/python-list
py3k: converting int to bytes
Is there a better way to convert int to bytes then going through strings: x=5 str(x).encode() Thanks. -- Yves. http://www.SollerS.ca/ http://blog.zioup.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
On 01/-10/-28163 02:59 PM, grobs456 wrote: #I set my system environment variables to: VARIABLEVALUE PYTHON_HOME c:\python27\python.exe PATH...;%PYTHON_HOME% #after also trying this: VARIABLEVALUE PYTHON_HOME c:\python27 PATH...;%PYTHON_HOME% The latter should be correct. But how are you actually setting them? In the Control Panel? And have you checked that they're actually set, by typing PATH at the command prompt, of a newly created DOS box? The path should consist of a number of directory locations, separated by semicolons (this is Windows). DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Python C Extensions
On 24/02/2011 16:01, aken8...@yahoo.com wrote: Hi, I have a memory leak problem with my "C" extension module. My C module returns large dictionaries to python, and the dictionaries never get deleted, so the memory for my program keeps growing. I do not know how to delete the dictionary object after it becomes irrelevant. I do not know if the version of python is relevant, I'm using the 2.5 ! Here is the "C" code: PyObject *GetDictionary(PyObject *self, PyObject *args) { PyObject *dict = PyDict_New(); PyObject *key; PyObject *value; char name[128]; for(int i = 0; i< 60; i++) { sprintf(name,"v%d",i); float number = 1.0 * 0.5*i; PyDict_SetItem(dict,Py_BuildValue("s",name),Py_BuildValue("f",number)); } return dict; } And here is the Code that I use in a loop, which causes the program memory to grow: import libpyTestModule as pyTEST bankTEST = {} for j in range(1,10): for k in range(1,10): bankTEST = pyTEST.GetDictionary() del bankTEST Any help will be appreciated. Py_BuildValue(...) returns an object with its refcount set to 1. PyDict_SetItem(...) increments the refcounts of the key and value objects when they are added to the dict, so their refcounts will then be 2. When the dict is garbage-collected the refcouts of the key and value objects will be decremented to 1, so they won't be collected, and as there aren't any other references to them, leading to a memory leak. You therefore need to decrement the refcounts of the key and value objects after adding them to the dict: PyObject *key = Py_BuildValue("s", name); PyObject *value = Py_BuildValue("f", number); PyDict_SetItem(dict, key, value); Py_DECREF(key); Py_DECREF(value); -- http://mail.python.org/mailman/listinfo/python-list
What is the preferred way to sort/compare custom objects?
I just discovered the wiki page on sorting (http://wiki.python.org/moin/HowTo/Sorting/). This describes the new way of sorting a container instead of using the cmp function. But what do I do for custom objects? If I write __lt__, __gt__, etc. functions for my objects, will these be used? Is this better than defining a key for sorting my custom objects? Thanks, Jeremy -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On 2/24/11 5:55 AM, Steven D'Aprano wrote: On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote: The IEEE 754 compliant FPU on most machines today, though, has an 80-bit internal representation. If you do a sequence of operations that retain all the intermediate results in the FPU registers, you get 16 more bits of precision than if you store after each operation. That's a big if though. Which languages support such a thing? C doubles are 64 bit, same as Python. C double *variables* are, but as John suggests, C compilers are allowed (to my knowledge) to keep intermediate results of an expression in the larger-precision FPU registers. The final result does get shoved back into a 64-bit double when it is at last assigned back to a variable or passed to a function that takes a double. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
n00b formatting
hi, all i can't believe i don't see this, but python from the command line: >>> x = '0D' >>> y = '0x' + x >>> print "%d" % int(y,0) 13 content of testme.py: x = '0D' y = '0x' + x print "%d" % int(y,0) TypeError: 'int' object is not callable what am i not seeing here?? -- http://mail.python.org/mailman/listinfo/python-list
Preferred method of sorting/comparing custom objects
I recently found the wiki page on sorting (http://wiki.python.org/moin/HowTo/Sorting/). This page describes the new key parameter to the sort and sorted functions. What about custom objects? Can I just write __lt__, __gt__, etc. functions and not have to worry about the key parameter? Is that the preferred (i.e., fastest) way to do things or should I use a lambda function similar to what is given as an example on the wiki page? For my custom objects, I would prefer to write the comparison functions as that seems easiest in my situation, but I would like to know what is the preferred/accepted way. Thanks, Jeremy -- http://mail.python.org/mailman/listinfo/python-list
Re: Python C Extensions
Thank you very much, it worked. I thought the PyDict_SetItem should assume ownership of the passed object and decrease it's reference count (I do not know why). Does this also go for the Lists ? Should anything inserted into list also be DECRED-ed ? Thank you again for reply. On Feb 24, 11:33 am, MRAB wrote: > On 24/02/2011 16:01, aken8...@yahoo.com wrote: > > > > > > > Hi, > > > I have a memory leak problem with my "C" extension module. My C module > > returns large dictionaries to python, and the dictionaries never get > > deleted, so the memory for my program keeps growing. > > > I do not know how to delete the dictionary object after it becomes > > irrelevant. I do not know if the version of python is relevant, I'm > > using the 2.5 ! > > > Here is the "C" code: > > > PyObject *GetDictionary(PyObject *self, PyObject *args) > > { > > PyObject *dict = PyDict_New(); > > PyObject *key; > > PyObject *value; > > > char name[128]; > > > for(int i = 0; i< 60; i++) > > { > > sprintf(name,"v%d",i); > > float number = 1.0 * 0.5*i; > > > PyDict_SetItem(dict,Py_BuildValue("s",name),Py_BuildValue("f",number)); > > } > > return dict; > > } > > > And here is the Code that I use in a loop, which causes the program > > memory to grow: > > import libpyTestModule as pyTEST > > > bankTEST = {} > > for j in range(1,10): > > for k in range(1,10): > > bankTEST = pyTEST.GetDictionary() > > del bankTEST > > > Any help will be appreciated. > > Py_BuildValue(...) returns an object with its refcount set to 1. > > PyDict_SetItem(...) increments the refcounts of the key and value > objects when they are added to the dict, so their refcounts will then > be 2. > > When the dict is garbage-collected the refcouts of the key and value > objects will be decremented to 1, so they won't be collected, and as > there aren't any other references to them, leading to a memory leak. > > You therefore need to decrement the refcounts of the key and value > objects after adding them to the dict: > > PyObject *key = Py_BuildValue("s", name); > PyObject *value = Py_BuildValue("f", number); > PyDict_SetItem(dict, key, value); > Py_DECREF(key); > Py_DECREF(value); -- http://mail.python.org/mailman/listinfo/python-list
Re: What is the preferred way to sort/compare custom objects?
Sorry for double posting. Google Groups was acting funny this morning. -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
On Feb 24, 11:32 am, Dave Angel wrote: > On 01/-10/-28163 02:59 PM, grobs456 wrote: > > > > > #I set my system environment variables to: > > > VARIABLE VALUE > > PYTHON_HOME c:\python27\python.exe > > PATH ...;%PYTHON_HOME% > > > #after also trying this: > > VARIABLE VALUE > > PYTHON_HOME c:\python27 > > PATH ...;%PYTHON_HOME% > > The latter should be correct. But how are you actually setting them? > In the Control Panel? And have you checked that they're actually set, > by typing > PATH > > at the command prompt, of a newly created DOS box? The path should > consist of a number of directory locations, separated by semicolons > (this is Windows). > > DaveA The issue was that I needed a new DOS box. Thanks everyone! -- http://mail.python.org/mailman/listinfo/python-list
Re: Preferred method of sorting/comparing custom objects
Sorry for double posting. Google Groups was acting funny this morning. -- http://mail.python.org/mailman/listinfo/python-list
Re: 2to3 chokes on bad character
On 2/24/2011 8:11 AM, Frank Millman wrote: future I will run some tests when betas are released, just in case I come up with something. Please do, perhaps more than once. The test suite coverage is being improved but is not 100%. The day *after* 3.2.0 was released, someone reported an unpleasant bug, a regression from 3.1.x. If they are tested with the last beta or first release candidate, it would have been found and fixed. Now its there until 3.2.1. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: What is the preferred way to sort/compare custom objects?
On Thu, Feb 24, 2011 at 8:27 AM, Jeremy wrote: > I just discovered the wiki page on sorting > (http://wiki.python.org/moin/HowTo/Sorting/). This describes the new way of > sorting a container instead of using the cmp function. But what do I do for > custom objects? > If I write __lt__, __gt__, etc. functions for my objects, will these be used? s/functions/methods/ Yes, they will. As Bill Nye would say: "Try It!". The REPL exists for a reason. If you're using Python 2.7+, you may want to use functools.total_ordering() [http://docs.python.org/library/functools.html#functools.total_ordering ] for convenience. > Is this better than defining a key for sorting my custom objects? Unless there are multiple "obvious" ways to sort your objects, yes. Third-party code will be able to sort+compare your objects. Sorting your objects in your own code will be more concise. And you'll be able to use the comparison operators on your objects. Note that, internally, it may be convenient to define the comparison methods in terms of a key comparison. For example: class Person(object): def __init__(self, first, last): self.first_name = first self.last_name = last def _key(self): return (self.last_name, self.first_name) def __lt__(self, other): return self._key() < other._key() # continue defining other comparison methods analogously... Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Why this difference?
On 2/24/2011 7:19 AM, n00m wrote: file my.txt: === 0 beb 1 qwe 2 asd 3 hyu 4 zed 5 asd 6 oth = py script: === import sys sys.stdin = open('88.txt', 'r') t = sys.stdin.readlines() t = map(lambda rec: rec.split(), t) print t print t[2][1] == t[5][1] print t[2][1] is t[5][1] print '==' a = 'asd' b = 'asd' print a is b output: === [['0', 'beb'], ['1', 'qwe'], ['2', 'asd'], ['3', 'hyu'], ['4', 'zed'], ['5', 'as d'], ['6', 'oth']] True False == True An implementation may *optionally* cache immutable values -- and change its internal rules as it pleases. When creating string objects from literals that look like identifiers, CPython does this, but apparently not when splitting an existing string. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Preferred method of sorting/comparing custom objects
On Thu, Feb 24, 2011 at 8:41 AM, Jeremy wrote: > I recently found the wiki page on sorting > (http://wiki.python.org/moin/HowTo/Sorting/). This page describes the new > key parameter to the sort and sorted functions. > > What about custom objects? Can I just write __lt__, __gt__, etc. functions > and not have to worry about the key parameter? Is that the preferred (i.e., > fastest) way to do things or should I use a lambda function similar to what > is given as an example on the wiki page? > > For my custom objects, I would prefer to write the comparison functions as > that seems easiest in my situation, but I would like to know what is the > preferred/accepted way. > > Thanks, > Jeremy Please don't make duplicate posts to the mailinglist/newsgroup. If you need to make corrections to a posting, do it as a follow-up to the original post. - Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b formatting
On Thu, Feb 24, 2011 at 8:41 AM, Verde Denim wrote: > hi, all > i can't believe i don't see this, but > python from the command line: x = '0D' y = '0x' + x print "%d" % int(y,0) > 13 > > content of testme.py: Is this the *entire* contents of the file? I suspect not, and that somewhere in it you assign to `int` as a variable, which should not be done since it's the name of the built-in type. > x = '0D' > y = '0x' + x > print "%d" % int(y,0) > TypeError: 'int' object is not callable > > what am i not seeing here?? Please include the full exception Traceback. Also, add: print type(int), int just before the existing `print`. Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Python C Extensions
On Feb 24, 8:46 am, "aken8...@yahoo.com" wrote: > Thank you very much, it worked. > I thought the PyDict_SetItem should assume ownership > of the passed object and decrease it's reference count (I do not know > why). > > Does this also go for the Lists ? Should anything inserted into list > also > be DECRED-ed ? The Python C API documentation has this information--if a function is documented as borrowing a reference, then it behaves as you were expecting (it doesn't increase the reference count). If it's documented as creating a new reference, it does increase the reference count. I don't know if there's a simple rule to know of a function borrows or creates a new reference; I've never noticed one. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k: converting int to bytes
On 2/24/2011 11:19 AM, s...@uce.gov wrote: Is there a better way to convert int to bytes then going through strings: x=5 str(x).encode() (This being Py3) If 0 <= x <= 9, bytes((ord('0')+n,)) will work. Otherwise, no. You would have to do the same thing str(int) does, which is to reverse the sequence of remainders from dividing by 10 and then add ord('0') to get the char code. Note: an as yet undocumented feature of bytes (at least in Py3) is that bytes(count) == bytes()*count == b'\x00'*count. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Preferred method of sorting/comparing custom objects
On Thu, Feb 24, 2011 at 9:41 AM, Jeremy wrote: > I recently found the wiki page on sorting > (http://wiki.python.org/moin/HowTo/Sorting/). This page describes the new > key parameter to the sort and sorted functions. > > What about custom objects? Can I just write __lt__, __gt__, etc. functions > and not have to worry about the key parameter? Is that the preferred (i.e., > fastest) way to do things or should I use a lambda function similar to what > is given as an example on the wiki page? > > For my custom objects, I would prefer to write the comparison functions as > that seems easiest in my situation, but I would like to know what is the > preferred/accepted way. If your objects naturally present a partial ordering, then you should implement it. If they do not, or if your implementations of the rich comparison functions would seem arbitrary in other contexts, then you should not add them to the classes, and you should use a key function instead. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python C Extensions
On 24/02/2011 16:46, aken8...@yahoo.com wrote: Thank you very much, it worked. I thought the PyDict_SetItem should assume ownership of the passed object and decrease it's reference count (I do not know why). Does this also go for the Lists ? Should anything inserted into list also be DECRED-ed ? Thank you again for reply. [snip] The pattern is that calls which create an object will return that object with a refcount of 1, and calls which 'store' an object, for example, PyDict_SetItem(...) and PyList_Append(...) will increment the refcount of the stored object to ensure that it won't be garbage collected. When in doubt, try stepping through the code in a debugger. You'll see that storing an object will cause its refcount to be incremented. -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b formatting
On 24/02/2011 16:41, Verde Denim wrote: hi, all i can't believe i don't see this, but python from the command line: >>> x = '0D' >>> y = '0x' + x >>> print "%d" % int(y,0) 13 content of testme.py: x = '0D' y = '0x' + x print "%d" % int(y,0) TypeError: 'int' object is not callable what am i not seeing here?? I can only assume that at some point you assigned an int to 'int'. -- http://mail.python.org/mailman/listinfo/python-list
Re: What is the preferred way to sort/compare custom objects?
On Thursday, February 24, 2011 10:09:53 AM UTC-7, Chris Rebert wrote: > On Thu, Feb 24, 2011 at 8:27 AM, Jeremy wrote: > > I just discovered the wiki page on sorting > > (http://wiki.python.org/moin/HowTo/Sorting/). This describes the new way > > of sorting a container instead of using the cmp function. But what do I do > > for custom objects? > > If I write __lt__, __gt__, etc. functions for my objects, will these be > >used? > > s/functions/methods/ > Yes, they will. As Bill Nye would say: "Try It!". The REPL exists for a > reason. > > If you're using Python 2.7+, you may want to use > functools.total_ordering() > [http://docs.python.org/library/functools.html#functools.total_ordering > ] for convenience. > > > Is this better than defining a key for sorting my custom objects? > > Unless there are multiple "obvious" ways to sort your objects, yes. > Third-party code will be able to sort+compare your objects. Sorting > your objects in your own code will be more concise. And you'll be able > to use the comparison operators on your objects. I implemented __eq__ and __lt__ (and used functoos, thanks for the suggestion) and sorting works. Thanks for the help. Most importantly, I wanted to make sure I was doing this the right way and your post helped. Jeremy -- http://mail.python.org/mailman/listinfo/python-list
Newbie...
Hi all... I am new to this list so treat me gently... ;o) I use Python almost totally differently to the vast majority of people. I like "banging the metal". As I know of no other way to give my Python code away I thought I`d join here. I only use STANDARD Python, no extras... Here are a couple of snippets. The first generates SINE, SQUARE, TRIANGLE, SAWTOOTH+, SAWTOOTH-, PULSE+ and PULSE- signals at the speakers/earphone socket(s). The second is a very simple DEMO which I will use in an attempt as an AudioScope using the external mic socket(s)... These are Public Domain and I hope they will be of interest to anyone. WATCH FOR INCORRECT INDEXING AND WORD WRAPPING... Enjoy finding simple solutions to often very difficult problems. == 1) The Function Generator... # A fun Python program to generate a Sine, Square, Triangle, Sawtooth, # and Pulse waveforms using STANDARD Python. # This is for (PC)Linux(OS), (ONLY?), and was done purely for fun. # (It now works on Debian 6.0.0 and Knoppix 5.1.1!) # # Although it is fairly easy to alter the frequency within limits I # have left it at approximately 1KHz to keep the code size down... # # It would be tricky to change the output level using STANDARD Python # for (PC)Linux(OS) 2009 using this idea to generate waveforms, but it # is possible within limits. # # Original idea copyright, (C)2009, B.Walker, G0LCU. # # DONATED TO LXF AS PUBLIC DOMAIN... # # It is assumed that /dev/audio exists; if NOT, then install oss-compat # from the distro`s repository. # # Ensure the sound system is not already in use. # # Copy the file to the Lib folder/drawer/directory where Python resides. # For a quick way to run just use at the ">>>" prompt:- # # >>> import afg[RETURN/ENTER] # # And away we go... # # The waveforms generated are unfiltered and therefore not pure, # but hey, a function generator, for free, without external hardware, # AND using standard Python, what more do you want... :) # Using my notebook about 150mV p-p was generated per channel at the # earphone socket(s). # # This code is Public Domain and you may do with it as you please... # # Coded on a(n) HP dual core notebook running PCLinuxOS 2009 and # Python 2.5.2 for Linux... # # You will need an oscilloscope connected to the earphone socket(s) # to see the resultant waveform(s) generated, or listen to the # harshness of the sound... ;o) # # It is EASILY possible to generate pseudo-random noise also but # I'll leave that for you to work out... :) # Import any modules, ~sys~ is not rquired but added nevertheless. import sys import os # Clear a terminal window ready to run this program. print os.system("clear"),chr(13)," ",chr(13), # The program proper... def main(): # Make all variables global, a quirk of mine... :) global sine global square global triangle global sawtoothplus global sawtoothminus global pulseplus global pulseminus global waveform global select global count # Allocate values to variables. # Any discrepancy between random soundcards may require small changes # in the numeric values inside each waveform mode... # These all oscillate at around 1KHz. sine=chr(15)+chr(45)+chr(63)+chr(45)+chr(15)+chr(3)+chr(0)+chr(3) square=chr(63)+chr(63)+chr(63)+chr(63)+chr(0)+chr(0)+chr(0)+chr(0) triangle=chr(0)+chr(7)+chr(15)+chr(29)+chr(63)+chr(29)+chr(15)+chr(7) sawtoothplus=chr(63)+chr(39)+chr(26)+chr(18)+chr(12)+chr(8)+chr(4)+chr(0) sawtoothminus=chr(0)+chr(4)+chr(8)+chr(12)+chr(18)+chr(26)+chr(39)+chr(63) pulseplus=chr(0)+chr(63)+chr(63)+chr(63)+chr(63)+chr(63)+chr(63)+chr(63) pulseminus=chr(63)+chr(0)+chr(0)+chr(0)+chr(0)+chr(0)+chr(0)+chr(0) # This is the INITIAL default waveform, the Square Wave. waveform=square select="G0LCU." count=1 # A continuous loop to change modes as required... while 1: # Set up a basic user window. print os.system("clear"),chr(13)," ",chr(13), print print "Simple Function Generator using STANDARD Python 2.5.2" print "for PCLinuxOS 2009, issued as Public Domain to LXF..." print print "Original idea copyright, (C)2009, B.Walker, G0LCU." print print "1) Sinewave." print "2) Squarewave." print "3) Triangle." print "4) Positive going sawtooth." print "5) Negative going sawtooth." print "6) Positive going pulse." print "7) Negative going pulse." print "Q) or q) to quit..." print # Enter a number
Re: 2to3 chokes on bad character
On Feb 25, 12:00 am, Peter Otten <__pete...@web.de> wrote: > John Machin wrote: > > Your Python 2.x code should be TESTED before you poke 2to3 at it. In > > this case just trying to run or import the offending code file would > > have given an informative syntax error (you have declared the .py file > > to be encoded in UTF-8 but it's not). > > The problem is that Python 2.x accepts arbitrary bytes in string constants. Ummm ... isn't that a bug? According to section 2.1.4 of the Python 2.7.1 Language Reference Manual: """The encoding is used for all lexical analysis, in particular to find the end of a string, and to interpret the contents of Unicode literals. String literals are converted to Unicode for syntactical analysis, then converted back to their original encoding before interpretation starts ...""" How do you reconcile "used for all lexical analysis" and "String literals are converted to Unicode for syntactical analysis" with the actual (astonishing to me) behaviour? -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie...
On 02/24/11 19:22, wisecrac...@tesco.net wrote: Hi all... I am new to this list so treat me gently... ;o) I for one welcome you :-) I use Python almost totally differently to the vast majority of people. I like "banging the metal". Well I can assure you that although you might be indeed in a minority, it is absolutely unlikely you are alone in that. As I know of no other way to give my Python code away I thought I`d join here. I'd would suggest you put it on a free repository like googlecode (http://code.google.com/) or my personal favourite bit bucket (https://bitbucket.org/) but there are many other out there too. Coincidentally both I have mentioned support Mercurial HG, which is largely written in Python too. And just post now and again a status update with links to the repository. Thanks for sharing and happy sharing! -- mph -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I define __getattr__ to operate on all items of container and pass arguments?
On Tuesday, February 15, 2011 2:58:11 PM UTC-7, Jeremy wrote: > > > So the arguments haven't yet been passed when __getattr__() is > > invoked. Instead, you must return a function from __getattr__(); this > > function will then get called with the arguments. Thus (untested): > > > > def __getattr__(self, name): > > def _multiplexed(*args, **kwargs): > > return [getattr(item, name)(*args, **kwargs) for item in self.items] > > return _multiplexed > Sorry to resurrect an old(ish) thread, but I have found a problem with this approach. Defining __getattr__ in this manner allows me to access methods of the contained objects---this was the original goal. But it has introduced a problem that I can no longer access the documentation for my class. The error I get is copied below. It seems like the help function is trying to access an attribute of the class, but can no longer get it. Any suggestions: Thanks, Jeremy /home/jlconlin/CustomPython/trunk/Collect.py in () > 1 2 3 4 5 /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.pyc in __call__(self, *args, **kwds) 455 def __call__(self, *args, **kwds): 456 import pydoc --> 457 return pydoc.help(*args, **kwds) 458 459 def sethelper(): /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pydoc.pyc in __call__(self, request) 1721 def __call__(self, request=None): 1722 if request is not None: -> 1723 self.help(request) 1724 else: 1725 self.intro() /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pydoc.pyc in help(self, request) 1768 elif request: doc(request, 'Help on %s:') 1769 elif isinstance(request, Helper): self() -> 1770 else: doc(request, 'Help on %s:') 1771 self.output.write('\n') 1772 /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pydoc.pyc in doc(thing, title, forceload) 1506 """Display text documentation, given an object or a path to an object.""" 1507 try: -> 1508 pager(render_doc(thing, title, forceload)) 1509 except (ImportError, ErrorDuringImport), value: 1510 print value l/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pydoc.pyc in render_doc(thing, title, forceload) 1483 desc = describe(object) 1484 module = inspect.getmodule(object) -> 1485 if name and '.' in name: 1486 desc += ' in ' + name[:name.rfind('.')] 1487 elif module and module is not object: TypeError: argument of type 'function' is not iterable -- http://mail.python.org/mailman/listinfo/python-list
share info between methods in a Class
What is the best way to access/change list or dictionary information between methods in the same class? Note : I would normally have the input_list originally populate by reading in a file. Currently I have: class Action: def __init__(self): self.input_List = list() self.output_List = list() self.ID_Name_Dict = dict() self.ID_Serial_Dict = dict() def get_Info(self, bldg=None): self.input_List = '111 Tom xyz-001 Enabled', '222 Dick xyz-002 Disabled', '333 Harry xyz-003 Enabled', \ '444 Jane abc-999 Disabled', '555 Holly xyz-999 Enabled', '666 Mary xyz-004 Enabled' for line in self.input_List: if line.find('Disabled') <= 0: item = line.split() ID = item[0] Name= item[1] Serial = item[2] self.ID_Name_Dict[ID] = Name self.ID_Serial_Dict[ID] = Serial def parse_Info(self, bldg=None): for each in self.ID_Name_Dict: if each.find('xyz') >= 0 and each.find('-999') <= 0: self.input_List.append(each) else: self.output_List.append(each) If this is not correct, can anyone provide some guidance? thanks, pg -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie...
Hi Martin... >> I am new to this list so treat me gently... ;o) > I for one welcome you :-) Many thanks... > Thanks for sharing and happy sharing! Now working on a simple home made seismometer and single channel audio oscilloscope using the recording technique previously posted, all in standard Python text mode... ;) And again thanks for the reply... Rigging for silent running... CYA... -- 73... Bazza, G0LCU... Team AMIGA... http://homepages.tesco.net/wisecracker/ http://main.aminet.net/search?readme=wisecracker http://mikeos.berlios.de/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why this difference?
@nn, @Terry Reedy: Good reading. Thanks. In fact now the case is closed. -- http://mail.python.org/mailman/listinfo/python-list
Suggestion Reqd for Designing a Website in Python
Dear Group, I have developed one big Machine Learning software a Machine Translation system in Python. Now, I am thinking to make a User Interface of it and upload it in a web site. My questions are: (i) For Designing an interface I am choosing Tkinter. Is it fine? (ii) How to connect this interface with my Tkinter based interface - should I have to recode the whole system? (iii) After designing I want to upload now how to do the server side scripting? If anyone can suggest it would be of great help. Thanks in advance, Best Regards, Subhabrata. -- http://mail.python.org/mailman/listinfo/python-list
Windows 7 Environment Setup - Python, R, mySQL
I am following along with this tutorial/post: http://www.walkerjeff.com/2011/02/python-to-r-via-mysql-on-windows-7-x64/ I am visualizing trying to setup some type of real world data/ interactions and stumbled upon this article. I am also on a 64bit Windows 7 machine. I am rusty in a sense as I have primarily been doing SQL Development/ Reporting on a DW and QA work for the past 3 years since graduating. I could use some support if anyone is available. I am prepared to provide any info one needs to better understand where I stand. I have begun fiddling with Python and R and am ready to re-install MySQL when I am sure of which version to get. Here is my environment/setup: Directories I am working through tutorials at, saving source code, etc. C:\dev\r C:\dev\python Python/R: C:\Python27 C:\Program Files\R\R-2.12.1 Questions: MySQL-python-1.2.1.tar.gz is for Linux, am I overlooking a Windows version? Should I install MySQL first? an earlier version of Python? I am not sure which file I should download with regards to RMySQL. Any assistance and advice would be highly valued. I am obviously first trying to get the environment up and running(which is what is overwhelming me a bit), and then come up with a real world scenarios of some sort that would enable me to play with all 3 languages to better understand the interactions/possibilities. I feel this would be a good way to dive into Python as I love being connected to a database and R would provide the option to graphically display the data in a multitude of formats and or gather statistics. I have a job interview coming up and I want to bang some nails before it. Kind Regards. -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b formatting
On Thu, Feb 24, 2011 at 12:49 PM, MRAB wrote: > On 24/02/2011 16:41, Verde Denim wrote: > >> hi, all >> i can't believe i don't see this, but >> python from the command line: >> >>> x = '0D' >> >>> y = '0x' + x >> >>> print "%d" % int(y,0) >> 13 >> >> content of testme.py: >> x = '0D' >> y = '0x' + x >> print "%d" % int(y,0) >> TypeError: 'int' object is not callable >> >> what am i not seeing here?? >> >> I can only assume that at some point you assigned an int to 'int'. > To put this in context, here is the code - #!/usr/bin/env python from scapy.all import * from binascii import hexlify import sys, re, pprint class C12Pcap: __acse_pdu_open = [] # 1-byte indicating start of packet __acse_pdu_len = [] # 1-byte indicating the length of the packet __asn1_called_ident_type = [] # 1-byte indicating universal identifier __asn1_called_ident_length = [] # 1-byte indicating the length of this identifier def __init__(self, pcap=None, portno=0): self.__pcap = pcap self.__portno = portno self.__pcktList = [] def __action(self): if self.__pcap: self.__rdpcap() self.__dump() return def __rdpcap(self): self.__pcktList = rdpcap(self.__pcap) return def __dump(self): x = int = 0 z = int = -1 for pckt in self.__pcktList: # for each packet z += 1 # debug a smaller subset... if z == 50: break # debug a smaller subset... layers = [] # a list of the layers' classes layerDicts = [] # a list of each layer's fields and values cltype = pckt.__class__ cl = pckt flds = cl.__dict__['fields'] while 1: # walk down the layers until no payload layers.append(cltype) layerDicts.append(flds) cltype = cl.__dict__['payload'].__class__ cl = cl.__dict__['payload'] flds = cl.__dict__['fields'] # if tcp, we'll guess at req/resp and if psh is on (for now) if re.search('TCP', str(cltype)): i = 0 for key, value in flds.items(): if key == 'flags' and long(value) == 24: # PUSH,ACK i = 1 if i == 1 and key == 'dport' and str(value) == str(portno): pktType = 'REQUEST' if i == 1 and key == 'sport' and str(value) == str(portno): pktType = 'RESPONSE' # Do we have a Raw packet - the interesting ones for us if re.search('Raw', str(cltype)): h = hexlify(flds['load']) # hex representation of payload self.__acse_pdu_open = h[0:2] self.__acse_pdu_len = h[2:4] self.__asn1_called_ident_type = h[4:6] self.__asn1_called_ident_length = '0x' + h[6:8] print self.__asn1_called_ident_length # WORKS FINE print "%d" % (int(self.__asn1_called_ident_length,0)) # FAILS WITH: #TypeError: 'int' object is not callable #File "/home/Scripts/Py/Elster_Lab/strip_raw_data1.py", line 103, in #inst.run(pcap,portno) #File "/home/Scripts/Py/Elster_Lab/strip_raw_data1.py", line 92, in run #self.__action() #File "/home/Scripts/Py/Elster_Lab/strip_raw_data1.py", line 41, in __action #self.__dump() #File "/home/Scripts/Py/Elster_Lab/strip_raw_data1.py", line 86, in __dump #print "%d" % (int(self.__asn1_called_ident_length,0)) if 'NoPayload' in str(cltype): break def run(self,pcap,portno): self.__pcap = pcap self.__portno = portno self.__action() def main(): return 0 if __name__ == '__main__': inst = C12Pcap() argl = sys.argv pcap = argl[1] portno = argl[2] inst.run(pcap,portno) main() -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b formatting
On Thu, Feb 24, 2011 at 12:23 PM, Chris Rebert wrote: > On Thu, Feb 24, 2011 at 8:41 AM, Verde Denim wrote: > > hi, all > > i can't believe i don't see this, but > > python from the command line: > x = '0D' > y = '0x' + x > print "%d" % int(y,0) > > 13 > > > > content of testme.py: > > Is this the *entire* contents of the file? I suspect not, and that > somewhere in it you assign to `int` as a variable, which should not be > done since it's the name of the built-in type. > > > x = '0D' > > y = '0x' + x > > print "%d" % int(y,0) > > TypeError: 'int' object is not callable > > > > what am i not seeing here?? > > Please include the full exception Traceback. > Also, add: > > print type(int), int > > just before the existing `print`. > > Cheers, > Chris > Found it... I can't believe I did this in the code... x = int = 0 ... Clobbered 'int' ... i hate n00b mistakes. -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b formatting
On 02/24/2011 04:46 PM, Verde Denim wrote: On Thu, Feb 24, 2011 at 12:49 PM, MRAB wrote: On 24/02/2011 16:41, Verde Denim wrote: x = '0D' y = '0x' + x print "%d" % int(y,0) TypeError: 'int' object is not callable what am i not seeing here?? I can only assume that at some point you assigned an int to 'int'. def __dump(self): x = int = 0 z = int = -1 As MRAB writes, you're assigning int values to "int" here, overriding the int() function. These are semi-equivalent to x = 0 int = 0 z = -1 int = 0 so when you get to print "%d" % (int(self.__asn1_called_ident_length,0)) "int" no longer refers to the built-in, but rather a name for "-1", thus the error. -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Why this difference?
On Thu, 24 Feb 2011 13:58:28 +0100, Paul Anton Letnes wrote: > Den 24.02.11 13.41, skrev n00m: >> The 1st "False" is not surprising for me. It's the 2nd "True" is a bit >> hmmm... ok, it doesn't matter == >> Have a nice day! > > I am no expert, but I think python re-uses some integer and string > objects. For instance, if you create the object int(2) it may be re-used > later if you have several 2 objects in your code. This is to save some > memory, or some other performance hack. Don't rely on it. Absolutely correct. It can be quite surprising when Python re-uses objects. E.g this surprised me: >>> x, y = "hello world", "hello world" >>> x == y, x is y (True, True) compared to this: >>> x = "hello world" >>> y = "hello world" >>> x == y, x is y (True, False) Don't rely on *any* of this, it's subject to change without notice. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On Thu, 24 Feb 2011 10:40:45 -0600, Robert Kern wrote: > On 2/24/11 5:55 AM, Steven D'Aprano wrote: >> On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote: >> >>> The IEEE 754 compliant FPU on most machines today, though, has an >>> 80-bit internal representation. If you do a sequence of operations >>> that retain all the intermediate results in the FPU registers, you get >>> 16 more bits of precision than if you store after each operation. >> >> That's a big if though. Which languages support such a thing? C doubles >> are 64 bit, same as Python. > > C double *variables* are, but as John suggests, C compilers are allowed > (to my knowledge) to keep intermediate results of an expression in the > larger-precision FPU registers. The final result does get shoved back > into a 64-bit double when it is at last assigned back to a variable or > passed to a function that takes a double. So... (1) you can't rely on it, because it's only "allowed" and not mandatory; (2) you may or may not have any control over whether or not it happens; (3) it only works for calculations that are simple enough to fit in a single expression; and (4) we could say the same thing about Python -- there's no prohibition on Python using extended precision when performing intermediate results, so it too could be said to be "allowed". It seems rather unfair to me to single Python out as somehow lacking (compared to which other languages?) and to gloss over the difficulties in "If you do a sequence of operations that retain all the intermediate results..." Yes, *if* you do so, you get more precision, but *how* do you do so? Such a thing will be language or even implementation dependent, and the implication that it just automatically happens without any effort seems dubious to me. But I could be wrong, of course. It may be that Python, alone of all modern high-level languages, fails to take advantage of 80-bit registers in FPUs *wink* -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie...
On Thu, 24 Feb 2011 19:22:52 +, wisecracker wrote: > As I know of no other way to give my Python code away I thought I`d join > here. It would be far more appropriate to *ask* where to put your code *first* rather than to just dump 350+ lines of double-spaced(!) code into people's inboxes, where it will likely be deleted and rapidly forgotten. The standard place for putting Python packages and modules is on the Python Package Index, PyPI: http://pypi.python.org/ For *small* snippets, say, a single function, you can use the ActiveState Cookbook: http://code.activestate.com/recipes/langs/python/ A few random comments about your code: > # Original idea copyright, (C)2009, B.Walker, G0LCU. You can't copyright ideas. > # >>> import afg[RETURN/ENTER] I thought you said you use only "STANDARD Python"? What's afg? It doesn't seem very standard to me: >>> import afg Traceback (most recent call last): File "", line 1, in ImportError: No module named afg > # Import any modules, ~sys~ is not rquired but added nevertheless. > import sys Don't do that. If you don't need a module, don't import it. *Especially* don't import it only to say "Hey, you don't need this code, I'm just wasting your time by making you read this!!!" > # The program proper... > def main(): > # Make all variables global, a quirk of mine... :) It's not 1970 any more. People will avoid like the plague code that over- uses globals. Also, there's no need to put code inside a "main" function that exists only to populate a few global values. Instead of this: def main(): global sine sine = 'blah blah blah' main() Just do this: sine = 'blah blah blah' See how much simpler and clearer it is? > sine=chr(15)+chr(45)+chr(63)+chr(45)+chr(15)+chr(3)+chr(0)+chr(3) This is much more easily and efficiently written as: sine = ''.join([chr(n) for n in (15, 45, 63, 45, 15, 3, 0, 3)]) or even shorter, as a string constant: sine = '\x0f-?-\x0f\x03\x00\x03' -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml
On Feb 24, 6:20 pm, Stefan Behnel wrote: > MRAB, 24.02.2011 01:25: > > The latest stable release is here: > > >http://pypi.python.org/pypi/lxml/2.2.8 > > Not quite the latest "stable release" (that would be 2.3), but at least one > that's pre-built for Windows. Christoph Gohlke has an 'unofficial' Window's binaries site that includes lxml 2.3 for Python 2.6, 2.7, 3.1 and 3.1: http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml Very handy site. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On Fri, 2011-02-25 at 00:33 +, Steven D'Aprano wrote: > On Thu, 24 Feb 2011 10:40:45 -0600, Robert Kern wrote: > > > On 2/24/11 5:55 AM, Steven D'Aprano wrote: > >> On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote: > >> > >>> The IEEE 754 compliant FPU on most machines today, though, has an > >>> 80-bit internal representation. If you do a sequence of operations > >>> that retain all the intermediate results in the FPU registers, you get > >>> 16 more bits of precision than if you store after each operation. > >> > >> That's a big if though. Which languages support such a thing? C doubles > >> are 64 bit, same as Python. > > > > C double *variables* are, but as John suggests, C compilers are allowed > > (to my knowledge) to keep intermediate results of an expression in the > > larger-precision FPU registers. The final result does get shoved back > > into a 64-bit double when it is at last assigned back to a variable or > > passed to a function that takes a double. > > So... > > (1) you can't rely on it, because it's only "allowed" and not mandatory; > > (2) you may or may not have any control over whether or not it happens; > > (3) it only works for calculations that are simple enough to fit in a > single expression; and > > (4) we could say the same thing about Python -- there's no prohibition on > Python using extended precision when performing intermediate results, so > it too could be said to be "allowed". > > > It seems rather unfair to me to single Python out as somehow lacking > (compared to which other languages?) and to gloss over the difficulties > in "If you do a sequence of operations that retain all the intermediate > results..." Yes, *if* you do so, you get more precision, but *how* do you > do so? Such a thing will be language or even implementation dependent, > and the implication that it just automatically happens without any effort > seems dubious to me. > > But I could be wrong, of course. It may be that Python, alone of all > modern high-level languages, fails to take advantage of 80-bit registers > in FPUs *wink* > > > > -- > Steven Maybe I'm wrong, but wouldn't compiling Python with a compiler that supports extended precision for intermediates allow Python to use extended precision for its immediates? Or does Python use its own floating-point math? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On 2011-02-25, Steven D'Aprano wrote: >> C double *variables* are, but as John suggests, C compilers are allowed >> (to my knowledge) to keep intermediate results of an expression in the >> larger-precision FPU registers. The final result does get shoved back >> into a 64-bit double when it is at last assigned back to a variable or >> passed to a function that takes a double. > > So... > > (1) you can't rely on it, because it's only "allowed" and not mandatory; > > (2) you may or may not have any control over whether or not it happens; > > (3) it only works for calculations that are simple enough to fit in a > single expression; and (3) is sort of an interesting one. If a C compiler could elminate stores to temporary variables (let's say inside a MAC loop) it might get a more accurate result by leaving temporary results in an FP register. But, IIRC the C standard says the compiler can only eliminate stores to variables if it doesn't change the output of the program. So I think the C standard actually forces the compiler to convert results to 64-bits at the points where a store to a temporary variable happens. It's still free to leave the result in an FP register, but it has to toss out the extra bits so that it gets the same result as it would have if the store/load took place. > (4) we could say the same thing about Python -- there's no > prohibition on Python using extended precision when performing > intermediate results, so it too could be said to be "allowed". Indeed. Though C-python _will_ (AFAIK) store results to variables everywhere the source code says to, and C is allowed to skip those stores, C is still required to produce the same results as if the store had been done. IOW, I don't see that there's any difference between Python and C either. -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On 2011-02-25, Westley Mart?nez wrote: > Maybe I'm wrong, but wouldn't compiling Python with a compiler that > supports extended precision for intermediates allow Python to use > extended precision for its immediates? I'm not sure what you mean by "immediates", but I don't think so. For the C compiler to do an optimization like we're talking about, you have to give it the entire expression in C for it to compile. From the POV of the C compiler, C-Python never does more than one FP operation at a time when evaluating Python bytecode, and there aren't any intemediate values to store. > Or does Python use its own floating-point math? No, but the C compiler has no way of knowing what the Python expression is. -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On Fri, 2011-02-25 at 00:57 +, Grant Edwards wrote: > On 2011-02-25, Westley Mart?nez wrote: > > > Maybe I'm wrong, but wouldn't compiling Python with a compiler that > > supports extended precision for intermediates allow Python to use > > extended precision for its immediates? > > I'm not sure what you mean by "immediates", but I don't think so. For > the C compiler to do an optimization like we're talking about, you > have to give it the entire expression in C for it to compile. From > the POV of the C compiler, C-Python never does more than one FP > operation at a time when evaluating Python bytecode, and there aren't > any intemediate values to store. > > > Or does Python use its own floating-point math? > > No, but the C compiler has no way of knowing what the Python > expression is. > > -- > Grant > > > I meant to say intermediate. I think I understand what you're saying. Regardless, the point is the same; floating-point numbers are different from real numbers and their limitations have to be taken into account when operating on them. -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k: converting int to bytes
On Feb 25, 4:39 am, Terry Reedy wrote: > Note: an as yet undocumented feature of bytes (at least in Py3) is that > bytes(count) == bytes()*count == b'\x00'*count. Python 3.1.3 docs for bytes() say same constructor args as for bytearray(); this says about the source parameter: """If it is an integer, the array will have that size and will be initialized with null bytes""" -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a safe use of eval?
On Thu, 24 Feb 2011 15:24:51 +0200, Frank Millman wrote: > Thanks, Christian. I had a look at that recipe, but I must say that Paul's > suggestion is much simpler - > >from ast import literal_eval >method_name = 'calc_area' >args = literal_eval('(100,200)') >result = getattr(my_inst, method_name)(*args) > > In my case the arguments are all strings or integers, so it looks as if this > approach should be safe. Do you see any problem with it? Only that you may need a fairly recent version of the ast module; the first attempt at literal_eval was a bit too ... literal, e.g. it couldn't handle negative numbers (Python doesn't have negative integer literals; evaluating "-10" applies the negation operator to the integer 10). -- http://mail.python.org/mailman/listinfo/python-list
Re: py3k: converting int to bytes
On 2/24/2011 9:25 PM, John Machin wrote: On Feb 25, 4:39 am, Terry Reedy wrote: Note: an as yet undocumented feature of bytes (at least in Py3) is that bytes(count) == bytes()*count == b'\x00'*count. Python 3.1.3 docs for bytes() say same constructor args as for bytearray(); this says about the source parameter: """If it is an integer, the array will have that size and will be initialized with null bytes""" Yes, it is there in the builtin functions section, but not in the doc strings. I opened an issue to fix the latter (#11310), with a proposed patch. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: wxPython SQLite and Reportlab demo
Beppe wrote: > >I would recommend this my little work on sourceforge. > > http://sourceforge.net/projects/pyggybank/ > >you can download an exe (pyggy_w32.7z) make with py2exe >and the source (pyggy_source.7z) >the project is named Pyggy Bank. Nowhere, in either this announcement or your sourceforge.net page, do you say a single word about what this application actually is. Just a few sentences describing what the application does would go a long way toward stirring up interest in this app. -- Tim Roberts, t...@probo.com Providenza & Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: Suggestion Reqd for Designing a Website in Python
joy99 wrote: > >Dear Group, >I have developed one big Machine Learning software a Machine >Translation system in Python. >Now, I am thinking to make a User Interface of it and upload it in a >web site. Do you mean you want people to download this from a web site as an executable, and then run it locally on their computers? Or do you mean you want people to use this as a web site, using their web browsers? The answers are very different. >My questions are: >(i) For Designing an interface I am choosing Tkinter. Is it fine? >(ii) How to connect this interface with my Tkinter based interface - >should I have to recode the whole system? >(iii) After designing I want to upload now how to do the server side >scripting? Question (i) implies that you want the first option (download an executable and run it). Question (iii) implies the second (that you want your app to run on a web server so people use it through your browser). Please clarify. -- Tim Roberts, t...@probo.com Providenza & Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml
MRAB, 24.02.2011 01:25: On 24/02/2011 00:10, Colin J. Williams wrote: Could someone please let me know whether lxml is available for Windows XP?. If so, is it available for Python 2.7? The latest stable release is here: http://pypi.python.org/pypi/lxml/2.2.8 Not quite the latest "stable release" (that would be 2.3), but at least one that's pre-built for Windows. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Is this a safe use of eval?
Hi all I know that the use of 'eval' is discouraged because of the dangers of executing untrusted code. Here is a variation that seems safe to me, but I could be missing something. I have a class, and the class has one or more methods which accept various arguments and return a result. I want to accept a method name and arguments in string form, and 'eval' it to get the result. Assume I have an instance called my_inst, and a method called 'calc_area', with arguments w and h. I then receive my_string = 'calc_area(100, 200)'. result = eval('my_inst.{0}'.format(my_string)) This will only work if the string contains a valid method name with valid arguments. Can anyone see anything wrong with this? Thanks Frank Millman -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a safe use of eval?
"Frank Millman" writes: > I then receive my_string = 'calc_area(100, 200)'. result = eval('my_inst.{0}'.format(my_string)) > This will only work if the string contains a valid method name with > valid arguments. > > Can anyone see anything wrong with this? Um, yes. What are valid arguments? Are you going to eval them? If they can only be literals, maybe you could use something like from ast import literal_eval method_name = 'calc_area' args = literal_eval('(100,200)') result = getattr(my_inst, method_name)(*args) but even that is risky in a hostile data environment. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python for embedded systems?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Aloha! On 2011:02:24 08:04 , Rafe Kettler wrote: > It's not a matter of language maturity, Python is very mature, it's a > matter of design. Python is a high-level, garbage-collected, > interpreted language, and that's not the ideal type of language for > embedded systems. Ideal, no. But usable and good to get a solution quckly, yes. I've developed and deployed a camera control system based on PC/104 running GNU/Linux and the application written in Python. The PC/104 fit the bill (money, size and power consumption) and the application performance in Python was more than required. Writing it in Python using modern tools talking to std devices in Linux made the development a snap. And the code is portable and easy to change. For really small embedded systems (8/16-bit MCUs) and hard RT-constraints Python is not a very good approach for the final system. But if you can fit a (possibly stripped down) standard OS such as Linux including Python, you get a very nice environment for a lot of embedded applications. Embedded is all about cost and hard constraints. But today you get quite a lot of compute resources for little money and in small form factors. Running applications on the bare metal, applications written in C is not always the optimal way to meet cost, technical and market requirements. Also, Python is a very nice language to model and explore the application and the design to be implemented. I use Python to build models of IP cores to be integrated into ASICs and FPGAs. Not using MyHDL for the actual core generation (yet) though. - -- Med vänlig hälsning, Yours Joachim Strömbergson - Alltid i harmonisk svängning. Kryptoblog - IT-säkerhet på svenska http://www.strombergson.com/kryptoblog -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk1mD4QACgkQZoPr8HT30QFgGwCg6fl4sTlUTF6ICSdC0nGy25hY NBgAnRjY+sZTFO3mjVsS0lsIjCz57qoK =FPHr -END PGP SIGNATURE- -- http://mail.python.org/mailman/listinfo/python-list