Re: Accelerating For Loop
Şansal Birbaş, 23.02.2011 07:34: I needed to find the cheapest combination among given data and I developed an algorithm for this task. It works correctly. But it takes much time (nearly 2 minutes) for second function to find the result while it is just one second for the first function. How can I improve the calculation speed? [...] for i in range(1,5): for j in i*range(max_expander+1): for k in i*range(max_expander+1): for m in i*range(max_expander+1): for r in i*range(max_expander+1): if (j+k+m+r)<=(i*max_expander): mevcut=zeros((1,5)) mevcut+=X*i mevcut+=Y*j mevcut+=Z*k mevcut+=T*m mevcut+=W*r m_UI=mevcut[0][0] m_UO=mevcut[0][1] m_AO=mevcut[0][2] m_BO=mevcut[0][3] i_AI=istenen[0] i_AO=istenen[1] i_BI=istenen[2] i_BO=istenen[3] if (m_UI>=(i_AI+i_BI)): if ((m_BO>=i_BO)): kalanUO=m_UO elif ((m_UO+m_BO)>=i_BO): kalanUO=(m_UO+m_BO)-i_BO if ((kalanUO+m_AO)>=i_AO): temp=[] temp.append(i) temp.append(j) temp.append(k) temp.append(m) temp.append(r) fiyat=i*X[4]+j*Y[4]+k*Z[4]+m*T[4]+r*W[4] temp.append(fiyat) combinations.append(temp) For this kind of computation, Cython is usually much faster than NumPy by itself. http://cython.org/ http://docs.cython.org/src/tutorial/numpy.html http://docs.cython.org/src/userguide/numpy_tutorial.html Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: How to use Python well?
Grant Edwards writes: > I've lost track of how many times I've tried to learn to use the Gnu > "info" command and gave up in frustration. I've never seen a program > with a more difficult to use UI. As I recall, there are other info viewers like tkinfo for example. But really, how hard is the basic info? Enter to go down a topic, u or l to back, s for search? Seems simple enough for me. What's really annoying is when the man page for a program finishes with a boilerplate announcement that the info version of the documentation is more complete. And yet, all the info version is, is the same manual page, including the same boilerplate... -- http://mail.python.org/mailman/listinfo/python-list
2to3 chokes on bad character
Hi all
I don't know if this counts as a bug in 2to3.py, but when I ran it on my
program directory it crashed, with a traceback but without any indication of
which file caused the problem.
Here is the traceback -
Traceback (most recent call last):
File "C:\Python32\Tools\Scripts\2to3.py", line 5, in
sys.exit(main("lib2to3.fixes"))
File "C:\Python32\lib\lib2to3\main.py", line 172, in main
options.processes)
File "C:\Python32\lib\lib2to3\refactor.py", line 700, in refactor
items, write, doctests_only)
File "C:\Python32\lib\lib2to3\refactor.py", line 294, in refactor
self.refactor_dir(dir_or_file, write, doctests_only)
File "C:\Python32\lib\lib2to3\refactor.py", line 314, in refactor_dir
self.refactor_file(fullname, write, doctests_only)
File "C:\Python32\lib\lib2to3\refactor.py", line 741, in refactor_file
*args, **kwargs)
File "C:\Python32\lib\lib2to3\refactor.py", line 336, in refactor_file
input, encoding = self._read_python_source(filename)
File "C:\Python32\lib\lib2to3\refactor.py", line 332, in
_read_python_source
return _from_system_newlines(f.read()), encoding
File "C:\Python32\lib\codecs.py", line 300, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 5055:
invalid start byte
On investigation, I found some funny characters in docstrings that I
copy/pasted from a pdf file.
Here are the details if they are of any use. Oddly, I found two instances
where characters 'look like' apostrophes when viewed in my text editor, but
one of them was accepted by 2to3 and the other caused the crash.
The one that was accepted consists of three bytes - 226, 128, 153 (as
reported by python 2.6) or 226, 8364, 8482 (as reported by python3.2).
The one that crashed consists of a single byte - 146 (python 2.6) or 8217
(python 3.2).
The issue is not that 2to3 should handle this correctly, but that it should
give a more informative error message to the unsuspecting user.
Frank Millman
BTW I have always waited for 'final releases' before upgrading in the past,
but this makes me realise the importance of checking out the beta versions -
I will do so in future.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Accelerating For Loop
Şansal Birbaş wrote: > for i in range(1,5): > for j in i*range(max_expander+1): > for k in i*range(max_expander+1): > for m in i*range(max_expander+1): > for r in i*range(max_expander+1): I must be missing something, but I can't see why you are using i*range(...) instead of just range(...). Can you enlighten me? -- http://mail.python.org/mailman/listinfo/python-list
Re: Is setdefaultencoding bad?
Ok, but that the interface handles UTF-8 strings are still ok? The defaultencoding is still ascii. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to use Python well?
Anssi Saari writes: > Grant Edwards writes: > > > I've lost track of how many times I've tried to learn to use the Gnu > > "info" command and gave up in frustration. I've never seen a program > > with a more difficult to use UI. > > As I recall, there are other info viewers like tkinfo for example. But > really, how hard is the basic info? Enter to go down a topic, u or l > to back, s for search? Seems simple enough for me. Yes, and non-standard; it ignores the finger-memory of both Vim *and* Emacs users. > What's really annoying is when the man page for a program finishes > with a boilerplate announcement that the info version of the > documentation is more complete. With the caveat “If the info and ls programs are properly installed at your site”, which it might not be; see below. > And yet, all the info version is, is the same manual page, including > the same boilerplate... That's what ‘info’ does if there's not actually any info documentation installed for the entry, but there is a manual page. Some of the GNU documentation is only licensed under FDL, which is (despite its name) a license that doesn't meet the FSF's own definition of software freedom. For that reason, it tends not to be included in some major free-software operating systems. A damned shame, but a result of some members in the FSF trying to apply different freedoms to different purposes. -- \ “I don't care to belong to a club that accepts people like me | `\as members.” —Groucho Marx | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess pipe question
This is a good solution thanks. You should wiki this somewhere. For extra points is there a way to speed up the p.stdout.read(bufsize) ? On Tue, Feb 22, 2011 at 7:57 PM, Rob Williscroft wrote: > Rita wrote in > news:AANLkTi=w95gxosc1tkt2bntgjqys1cbmdnojhokq4...@mail.gmail.com in > gmane.comp.python.general: > > > > > When using wait() it works a bit better but not consistent > > def run(cmd): > > p=subprocess.Popen(cmd,stdout=subprocess.PIPE) > > rc=p.wait() > > print rc > > return p.stdout > > > > > When the output of cmd is a small ascii file it works perfectly fine, > > but when the file is large (more than 2MB) the process just waits for > > ever (I am guessing its blocking?). > > Your OS has supplied a pipe buffer of 2MB, its full and the prossess > is waiting until you read something from the pipe (i.e. p.stdout.read()). > > >When I use the communicate call > > it works perfectly but my process is consuming way too much memory. > > > > Is there a better way to get my return code consistently efficiently > > and not take up so much memory? > > If you don't need the processes output then don't use the PIPE argument. > If you do you will need to read from p.stdout until the process is > complete, then get the return code: > > while True: >buf = p.stdout.read( 1024 ) ># do somthing with buf >if len( buf ) < 1024: >break > > rc = p.wait() > > Rob. > > -- > http://mail.python.org/mailman/listinfo/python-list > -- --- Get your facts first, then you can distort them as you please.-- -- http://mail.python.org/mailman/listinfo/python-list
Re: Is setdefaultencoding bad?
On Wed, Feb 23, 2011 at 3:07 AM, moerchendiser2k3 wrote: > Ok, but that the interface handles UTF-8 strings > are still ok? The defaultencoding is still ascii. Yes, that's fine. UTF-8 is an excellent encoding choice, and encoding/decoding should always be done explicitly in Python, so the "default encoding" ideally ought to never come into play (and indeed, Python 3 does away with bug-prone implicit encoding/decoding entirely FWICT). Having ASCII as the "default encoding" ensures that implicit encoding/decoding bugs are relatively apparent. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Yappi error "context not found"
On 10 Şubat, 08:38, Brian wrote: > I'm posting here because I can't find a Yappi specific mailing list. > > I've been using the rather brilliant Yappi fromhttp://code.google.com/p/yappi/ > It works well for small programs with a few threads. However, when > trying to run it over large programs with several hundred threads I've > been getting the error "context not found". > > The most basic example I can come up with which illustrates the > problem is below. Changing the number of threads started from 25 down > to, say, 10, works fine. Any more than ~25 and the error keeps > appearing. This is using Python 2.6.5 and Ubuntu Linux 10.04. I don't > have another box to try this out on at the moment. > > I've tested a similar script using thread.start_new_thread() and that > doesn't seem to have the problem. So is there some issue with Yappi > and inheriting from threading.Thread? > > import yappi > import time > import threading > > class MyThread(threading.Thread): > def run(self): > time.sleep(1) > > yappi.start() > > for i in range(0,25): > c = MyThread() > c.start() > time.sleep(1) > > yappi.print_stats() > yappi.stop() > > Running the above gives: > [*] [yappi-err] context not found. > [*] [yappi-err] context not found. > [*] [yappi-err] context not found. > [*] [yappi-err] context not found. > ... > > Any help appreciated. Can you open a ticket against this? This error is because yappi cannot seem to determine the actual created threads by the Python VM. This may be due to several things, but in our tests we have successully profiled more than 1000 threads simultaneously. You can open and follow an issue here: http://code.google.com/p/yappi/issues/list -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating Long Lists
On Tue, 2011-02-22, Ben Finney wrote: > Kelson Zawack writes: > >> I have a large (10gb) data file for which I want to parse each line >> into an object and then append this object to a list for sorting and >> further processing. > > What is the nature of the further processing? > > Does that further processing access the items sequentially? If so, they > don't all need to be in memory at once, and you can produce them with a > generator http://docs.python.org/glossary.html#term-generator>. He mentioned sorting them -- you need all of them for that. If that's the *only* such use, I'd experiment with writing them as sortable text to file, and run GNU sort (the Unix utility) on the file. It seems to have a clever file-backed sort algorithm. /Jorgen -- // Jorgen GrahnO o . -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating Long Lists
On Wed, 2011-02-23 at 13:57 +, Jorgen Grahn wrote: > If that's the *only* such use, I'd experiment with writing them as > sortable text to file, and run GNU sort (the Unix utility) on the file. > It seems to have a clever file-backed sort algorithm. +1 - and experiment with the different flags to sort (compression of intermediate results, intermediate batch size, etc) Tim -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On 2011-02-23, Terry Reedy wrote: > On 2/22/2011 2:42 PM, Grant Edwards wrote: > >> Except that Python (and computer languages in general) don't deal with >> real numbers. They deal with floating point numbers, which aren't the >> same thing. [In case anybody is still fuzzy about that.] > > In particular, floats are a fixed finite set of rationals with adjusted > definitions of the arithmetic operators. The adjustment is necessary > because the 'proper' answer to an operation may not be one of the > allowed answers. In other words, f1 float-op f2 may not be the same as > f1 rat-op f2, and hence float-ops do not always obey the rules of > rational (or real) operations. On some (increasingly rare) systems they don't always obey the rules of base-two float-opts either, but that's a whole different can of worms. -- Grant Edwards grant.b.edwardsYow! I want a VEGETARIAN at BURRITO to go ... with gmail.comEXTRA MSG!! -- http://mail.python.org/mailman/listinfo/python-list
Trouble installing numpy on QNX
I'm currently using QNX 6.4.1 with Python 2.5. I went to install numpy 1.4.1, but the install kicks bakc an error saying that it cannot find Python.h and that I should install python-dev|python-devel. I look online and I can only find those two packages in relation to Ubuntu, which obviously will not work. Has anyone experienced this or know what's up? -- http://mail.python.org/mailman/listinfo/python-list
Porting Python2 C-API/Swig based modules to Python 3
Hello,
I am trying to get a compiled module to work with Python3. The code I
am compiling was originally intended to be used in a Python 2.*
environment, but I updated all the Python 2.* elements and the swig
commands used by the setup.py script. I got the library to
successfully compile, but I am getting the following errors on import
(below). I am not sure how to trouble shoot this problem, and the
fact that only one symbol (_PyCObject_FromVoidPtr) is missing is
disconcerting. I Googled some, but the symbol mentioned only showed
up in a few posts, where linking was an issue. I have also gone
through the setup.py script and explicitly defined all the library
paths.
My questions:
- Has anyone ever ported Python 2* modules/libs to Python 3 that rely
on swig, and are there some changes in the C-API/swig I need to be
looking for to make this port successful?
- Does anyone have any advice/insght about how I can troubleshoot,
diagnose, and resolve this issue?
Thanks in advance,
-- Adam
0:pylibpcap-0.6.2$ python3
Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pcap,py
Traceback (most recent call last):
File "", line 1, in
File
"/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
line 25, in
_pcap = swig_import_helper()
File
"/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
line 21, in swig_import_helper
_mod = imp.load_module('_pcap', fp, pathname, description)
ImportError:
dlopen(/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so,
2): Symbol not found: _PyCObject_FromVoidPtr
Referenced from:
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
Expected in: flat namespace
in
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
>>> ^D
--
http://mail.python.org/mailman/listinfo/python-list
Re: Trouble installing numpy on QNX
On 2/23/11 10:16 AM, Nelson Powell wrote: I'm currently using QNX 6.4.1 with Python 2.5. I went to install numpy 1.4.1, but the install kicks bakc an error saying that it cannot find Python.h and that I should install python-dev|python-devel. I look online and I can only find those two packages in relation to Ubuntu, which obviously will not work. Has anyone experienced this or know what's up? Those are just the most common names for the packages that include the Python headers. You will have to consult the source of your Python binaries to find out how to get the headers installed. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
CentOS 5.5 x86_64 rpmbuild from source
When I run I'm attempting to run "rpmbuild -ba SPECS/python-2.7.spec" I get the following error: ERROR 0001: file '/usr/lib/python2.7/lib-dynload/_bsddb.so' contains a standard rpath '/usr/lib64' in [/usr/lib64] ERROR 0001: file '/usr/lib/python2.7/lib-dynload/_sqlite3.so' contains a standard rpath '/usr/lib64' in [/usr/lib64] error: Bad exit status from /var/tmp/rpm-tmp.24791 (%install) Any clues or suggestions on how I should tackle the issue? I feel lost in my limited capacity. Thanks, Will -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes inheritance issue
On Feb 23, 12:38 am, Carl Banks wrote: > Steve wrote: > > I've filed a bug in python but I wanted to see if other ctypes users/ > > experts viewed this issue as a bug. > > Consider the following: > > python code: > > import ctypes > > class my_array( ctypes.Array ): > > _type_ = ctypes.c_uint8 > > _length_ = 256 > > > class my_array2( my_array ): > > pass > > > Output: > > class my_array2( my_array ): > > AttributeError: class must define a '_length_' attribute, which must > > be a positive integer > > I'm not sure if I'd call it a bug, but I can tell you (probably) why it > happens. > > ctypes.Array uses a custom metaclass that inspects the class dictionary > directly for those attributes before passing them on to the type > constructor. In simple terms, it means that ctypes is using the > _length_ and _type_ attributes before it creates the class. > > That's an unfortunate aspect of metaclass programming: sometimes you > need that information in order to create the class, but you need the > class in order to inherit. So, you are stuck with forcing subtypes to > redefine the same attributes as their parents, or trying to mimic the > inheritance by searching the bases yourself (which can be prone to > mistakes). > > Point is: this behavior is probably the way it is because of a technical > difficulty as opposed to an oversight or design choice. > > > This is analogous to the C code > > typedef char my_array[ 256 ]; > > typedef my_array my_array2; > > Not really. Typedefs don't define new types in C; they define aliases > for existing types. This is what would be analogous to your C code: > > my_array_2 = my_array > > (Which is not to say this affects whether this is a bug or not.) > > > As shown above, the python code raises exceptions claiming _type_ and > > _length_ have not been defined. This seems like a bug. I shouldn't > > need to redefine _type_ and _length_, otherwise there was no point in > > subclassing from my_array. > > > Thoughts? > > I would vote "not a bug", though I'm pretty close to the fence. > > I doubt this behavior is documented one way or the other (if it is > that that would change things), so for it to be considered a bug it'd > have to be very contrary to user expectations. Not inheriting is quite > unexpected, yes, but when metaclass programming is involved all bets are > off about stuff like that. A lot of what makes metaclasses useful > depends on them breaking expectations with respect to ordinary > classes, so I don't think it would be wise to label every divergence a > bug. > > Carl Banks After looking at some metaclass examples it appears this information is readily available. A metaclass gets a dictionary containing information about the parent class (or should, at least). It seems like it must have this information in order to dynamically make decisions about how to create classes... So, "bug" or not, shouldn't this just work? Is there something that prevents it from being implemented? Would this break something? -- http://mail.python.org/mailman/listinfo/python-list
Re: Porting Python2 C-API/Swig based modules to Python 3
On Feb 23, 8:54 am, Adam Pridgen
wrote:
> Hello,
>
> I am trying to get a compiled module to work with Python3. The code I
> am compiling was originally intended to be used in a Python 2.*
> environment, but I updated all the Python 2.* elements and the swig
> commands used by the setup.py script. I got the library to
> successfully compile, but I am getting the following errors on import
> (below). I am not sure how to trouble shoot this problem, and the
> fact that only one symbol (_PyCObject_FromVoidPtr) is missing is
> disconcerting. I Googled some, but the symbol mentioned only showed
> up in a few posts, where linking was an issue. I have also gone
> through the setup.py script and explicitly defined all the library
> paths.
>
> My questions:
>
> - Has anyone ever ported Python 2* modules/libs to Python 3 that rely
> on swig, and are there some changes in the C-API/swig I need to be
> looking for to make this port successful?
>
> - Does anyone have any advice/insght about how I can troubleshoot,
> diagnose, and resolve this issue?
>
> Thanks in advance,
>
> -- Adam
>
>
>
> 0:pylibpcap-0.6.2$ python3
> Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.>>>
> import pcap,py
>
> Traceback (most recent call last):
> File "", line 1, in
> File
> "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
> line 25, in
> _pcap = swig_import_helper()
> File
> "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
> line 21, in swig_import_helper
> _mod = imp.load_module('_pcap', fp, pathname, description)
> ImportError:
> dlopen(/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so,
> 2): Symbol not found: _PyCObject_FromVoidPtr
> Referenced from:
> /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
> Expected in: flat namespace
> in
> /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
>
>
>
> >>> ^D- Hide quoted text -
>
> - Show quoted text -
This is a change in the C-API in 3.2. See http://bugs.python.org/issue5630
casevh
--
http://mail.python.org/mailman/listinfo/python-list
Securing files
Hello Folks, In a python application that I'm developing I've been asked to add security to databases that the program might create and access; the database is to be password protected by its creator. The application uses an SQLite database, which could be changed for another back-end if that would offer better security, but I would still like to use an embeddable database file. The problem isn't so much the database itself, as I can think of a number of way to encrypt the data it contains, but some of the data is simply names of image and video files contained elsewhere in the file-system. Is there anyway to prevent a user from simply opening up the file-system from outside of the application and viewing the files? One way that I can think of would be to encode the image/video files as BLOBS and store them in the database itself, but apart from that option, can anyone suggest other ways? I'm currently working with python2.7 under Windows7, but I'm hoping to extend the application to Linux and Mac also. Thank you for your help. Best regards, Tim -- http://mail.python.org/mailman/listinfo/python-list
Re: Securing files
On Wed, Feb 23, 2011 at 11:48 AM, Timothy W. Grove wrote: > Hello Folks, > > In a python application that I'm developing I've been asked to add security > to databases that the program might create and access; the database is to be > password protected by its creator. The application uses an SQLite database, > which could be changed for another back-end if that would offer better > security, but I would still like to use an embeddable database file. > > The problem isn't so much the database itself, as I can think of a number of > way to encrypt the data it contains, but some of the data is simply names of > image and video files contained elsewhere in the file-system. Is there > anyway to prevent a user from simply opening up the file-system from outside > of the application and viewing the files? One way that I can think of would > be to encode the image/video files as BLOBS and store them in the database > itself, but apart from that option, can anyone suggest other ways? I'm > currently working with python2.7 under Windows7, but I'm hoping to extend > the application to Linux and Mac also. Thank you for your help. I don't know much about your system, but just as a hunch I'd say that if this is a major requirement you're screwed. The old maxim: there is no cryptographic solution to the problem in which the attacker and intended recipient are the same person. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Securing files
In article , "Timothy W. Grove" wrote: > The problem isn't so much the database itself, as I can think of a > number of way to encrypt the data it contains, but some of the data is > simply names of image and video files contained elsewhere in the > file-system. Is there anyway to prevent a user from simply opening up > the file-system from outside of the application and viewing the files? > One way that I can think of would be to encode the image/video files as > BLOBS and store them in the database itself, but apart from that option, > can anyone suggest other ways? You could have your application create one-time URLs for the images, something like http://host.com/image/xxx/yyy, where xxx is a string identifying the particular image and yyy is a cryptographic key encoding not just xxx, but also a timestamp, and maybe a session key or a cookie identifying the client. The image URL is thus good for only a small time window, for a specific client. Naturally, you need to move the media files to someplace where they are not externally visible. And write the code to generate and check the crypto keys. You *could* store them as database blobs, but just moving them out of the externally visible directory tree would be easier, more efficient, and just as secure. None of this, of course, has anything to do with Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to compile on OS X PPC? was: Re: [RELEASED] Python 3.2
On 23-02-11 02:15, Ned Deily wrote: Unfortunately, this is a variation of an old issue that hasn't yet been fixed (http://bugs.python.org/issue1099). The simplest workaround is to include the --enable-universalsdk option to configure, so something like this: ./configure --enable-framework --enable-universalsdk=/ Thank you Ned! That did the trick. I'm not entirely sure what that option does (it seemed to fail when not specifying it as '/', and '/' seems an odd location to set it to if it's meant to be a filesystem path), but hey, the compilation succeeded. Another solution is to use the 3.2 32-bit installer for Mac OS X from python.org: http://www.python.org/download/releases/3.2/ Yep, maybe I should have. I just like to tinker around too much myself I suppose :-) Thanks again. Irmen de Jong -- http://mail.python.org/mailman/listinfo/python-list
Re: Python fails on math
On 2/22/2011 9:59 AM, Roy Smith wrote: In article, Grant Edwards wrote: Python doesn't do equations. Python does floating point operations. More generally, all general-purpose programming languages have the same problem. You'll see the same issues in Fortran, C, Java, Ruby, Pascal, etc, etc. Not quite. CPython has the problem that it "boxes" its floating point numbers. After each operation, the value is stored back into a 64-bit space. The IEEE 754 compliant FPU on most machines today, though, has an 80-bit internal representation. If you do a sequence of operations that retain all the intermediate results in the FPU registers, you get 16 more bits of precision than if you store after each operation. Rounding occurs when the 80-bit value is forced back to 64 bits. So it's quite possible that this would look like an equality in C, or ShedSkin, or maybe PyPy (which has some unboxing optimizations) but not in CPython. (That's not the problem here, of course. The problem is that the user doesn't understand floating point. The issues I'm talking about are subtle, and affect few people. Those of us who've had to worry about this and read Kahan's papers are typically developers of simulation systems, where cumulative error can be a problem. In the 1990s, I had to put a lot of work into this for collision detection algorithms for a physics engine. As two objects settle into contact, issues with tiny differences between large numbers start to dominate. It takes careful handling to prevent that from causing high frequency simulated vibration in the simulation, chewing up CPU time at best and causing simulations to fly apart at worst. The problems are understood now, but they weren't in the mid-1990s. The licensed Jurassic Park game "Trespasser" was a flop for that reason.) John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Running Scripts vs Interactive mode
Hi, I am trying to work through the tutorial at: http://docs.python.org/tutorial/ The issue I am facing is with regards to the discussion about "Invoking the Interpreter" and "Executable Python Scripts". It was rather hazy in my opinion. see: http://docs.python.org/tutorial/interpreter.html I realize I can double click on a .py file and Windows treats it as an executable but the command prompt pops in and out so quickly that I can't see the results of my script. I would prefer to follow along with the tutorial, compiling new ticks and topics into a growing .py file, that I error check of course(in case the same variables are used throughout the tutorial), and then run that file to see the results as if I would have typed each line in one at a time. This way I have something usable to refer back to for learning purposes and for future coding projects instead of having to rely on the tutorial itself for my reference moving forward. Any ideas? Python is installed at: C:\Python27 and I ran: set path=%path%;C:\python27 #do i have to run the above each time I open up a session? #Also, note that I do not see the any system environment variables get updated when I run the above. -- http://mail.python.org/mailman/listinfo/python-list
urllib2 header casing discrepancy
Spent some time tripping today over HTTP headers being added to the
Request object in "proper" header case (all components title cased)
but finding that the resulting header dictionary object had the
headers keyed with first component only capitalized.
## http://paste.pound-python.org/show/3273/
>>> import urllib2
>>> req = urllib2.Request("http://www.google.com";)
>>> req.add_header('User-Agent', 'bob')
>>> req.add_header('X-Poopy', 'Big Poopy')
>>> req.get_header('User-Agent')
>>> req.get_header('X-Poopy')
>>> req.headers
{'X-poopy': 'Big Poopy', 'User-agent': 'bob'}
>>> req.get_header('User-agent')
'bob'
>>> req.get_header('X-poopy')
'Big Poopy'
I'd expect the header put in as "User-Agent" to be stored and
retrieved using the same case but this is clearly not the case. HTTP
header names are case-insensitive so it matters not how they're sent
but programmatically it would seem like we'd want to ensure that the
specified input is preserved.
http://www.cs.tut.fi/~jkorpela/http.html
http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2
This is observed under Python 2.6.1 and 2.6.6 on OS X and Linux.
Am I misunderstanding how to use these functions? Do I need to adjust
my expectations? :)
--
Darren Spruell
phatbuck...@gmail.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: QWebView: notify python when I clicked on a certain web eleemnt
Hi Damjan, Thanks for your answer On 02/21/2011 05:09 PM, Дамјан Георгиевски wrote: >> I have a web page (existing page, can't modify it) and I would like >> to browse it in a QtWebview. (This is already working >> When I click on a certain element e.g. "" >> I would like to notify my python script. >> >> What is important: >> I don't want to change anything else in the web pages behaviour (This >> means if clicking on this element will call some java script >> functions, load a pag, then this should still happen. > > > You can inject JavaScript using the method > QWebElement::evaluateJavaScript() > > Also you can insert a Python object in the JS runtime with > QWebFrame::addToJavaScriptWindowObject() > see here: > http://pysnippet.blogspot.com/2010/01/calling-python-from-javascript-in- > pyqts.html > > > so make a Python object with some methods you'd wish to call, then > evaluate some small JS that will bind the click event on your element to > the method of the Python object. > OK, but how do I make sure, that I don't override an existing binding to the element's click() event? I will start playing with it. > -- http://mail.python.org/mailman/listinfo/python-list
Re: Securing files
Can't you just create a new user and a special group for him and limit access privileges to that precise folder only to that user/group? That way only "the mysterious application" would be able to access them. It's perfect and very simple solution. Storing especially video files (that are usually quite big) in a blob is not very efficient and would kill data base engine if someone would like to open big movie. Need more info. Is it online/web/server application? What's its main purpose. -- Greets, entliczek -- http://mail.python.org/mailman/listinfo/python-list
Re: QWebView: notify python when I clicked on a certain web eleemnt
On Wed, Feb 23, 2011 at 4:02 PM, Gelonida wrote: >> so make a Python object with some methods you'd wish to call, then >> evaluate some small JS that will bind the click event on your element to >> the method of the Python object. >> > > OK, but how do I make sure, that I don't override an existing binding to > the element's click() event? If the element has an existing binding, then create a new JS function that calls both the Python method and the original event handler, and bind that function to the event. -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
Add into each your *.py script and as the very last line this:
raw_input('Press any key to exit...')
Or, even better, I'd recommend this free Python Editor:
http://pythonide.stani.be/
SPE IDE - Stani's Python Editor
Free python IDE for Windows,Mac & Linux with UML,PyChecker,Debugger,
GUI design,Blender & more
A superb thing, imo.
But firstly download and install python package *wxPython*:
wxPython2.8-win32-unicode-2.8.11.0-py25.exe
SPE Editor needs it to work properly.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python programming books
"John Bokma" wrote in message news:87oc63nvuo@castleamber.com... I also like the "Python Essential Reference" a lot. I'd second that. "Python Essential Reference" effectively documents the "batteries included" aspect of Python, using lots of good examples in a quite readable tome. -- http://mail.python.org/mailman/listinfo/python-list
Python for embedded systems?
Apologies if this has been asked; I haven't yet Googled the archives. >From a brief email conversation, Guido pointed me to this newsgroup to ask the following questions: "Is Python 'mature' enough to be considered the primary language for embedded systems? Is the performance there for real-time applications (eg avionics, real-time control systems) or is it still more suitable "...as a glue language, used to combine components written in C++" ?" And further: "Has anyone tried to shorten development time when porting code to a new embedded hardware platform, by trying to convert legacy code (C/C+ +/Ada) to Python?" I'm currently thinking that Python isn't there yet but certainly would like to hear any feedback. Your input is greatly appreciated; thanks! - Paulito -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
grobs456 writes: > I am trying to work through the tutorial at: > http://docs.python.org/tutorial/ Excellent! I heartily encourage everyone to do this when learning Python: *work through* (not just read) the tutorial. Good for you! > I realize I can double click on a .py file and Windows treats it as an > executable but the command prompt pops in and out so quickly that I > can't see the results of my script. Don't do that, then :-) You can open a persistent terminal window, and run the program from the command line. > I would prefer to follow along with the tutorial, compiling new ticks > and topics into a growing .py file Not a good idea; the exercises aren't meant to be all run in an ever-growing single program. You should instead make a directory of small programs that are independent, one for each exercise. > Python is installed at: > C:\Python27 > > and I ran: > set path=%path%;C:\python27 > #do i have to run the above each time I open up a session? You should be able to set that value persistently. (I don't wash Windows, so I don't know what you need to do there.) > #Also, note that I do not see the any system environment variables > get updated when I run the above. Right, that's only for the current environment (generally, one program and any of its child processes). You'll need to find how to set environment variables in your login profile if you want that to persist. Good hunting. -- \“Good judgement comes from experience. Experience comes from | `\ bad judgement.” —Frederick P. Brooks | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
lxml
Could someone please let me know whether lxml is available for Windows XP?. If so, is it available for Python 2.7? Thanks, Colin W. -- http://mail.python.org/mailman/listinfo/python-list
Re: Porting Python2 C-API/Swig based modules to Python 3
Thanks for the help. That information got me started in the right
direction. I put my notes up on paste bin for others to use in the
future.
For the future reference of others, I have included the pertinent
details below. My process for identifying the various issues was
compiling and running the project to see what was still broken after
each fix. I know this is not the most efficient way, but it worked
for this pylibcap since the code base was pretty small. I don't think
everything is completely fixed, but its a step in the right direction.
The initial hurdle I encountered was swig. I had to modify swig's
code output, because it was not using the PyCapsule_* API yet. I
found someones solution here: http://paste.pocoo.org/show/253494/.
After applying that fix, I found that the pcap_interface.c and pcap.i
files in the project were using some deprecated API calls. So I
converted the following APIs:
PyLong_AS_INT ==> PyLong_AsLong
PyInt_* ==> PyLong_*
PyString_* ==> PyBytes_* (initialily changed to PyUnicode_* but that
was wrong, because I needed the Raw Bytes)
I also had to change s# to y#. Before, s# stood for string and #, but
it now means unicode and number.
Hopefully these details will be helpful to others in the future.
Thanks again for the nudge,
-- Adam.
On Wed, Feb 23, 2011 at 11:58 AM, casevh wrote:
> On Feb 23, 8:54 am, Adam Pridgen
> wrote:
>> Hello,
>>
>> I am trying to get a compiled module to work with Python3. The code I
>> am compiling was originally intended to be used in a Python 2.*
>> environment, but I updated all the Python 2.* elements and the swig
>> commands used by the setup.py script. I got the library to
>> successfully compile, but I am getting the following errors on import
>> (below). I am not sure how to trouble shoot this problem, and the
>> fact that only one symbol (_PyCObject_FromVoidPtr) is missing is
>> disconcerting. I Googled some, but the symbol mentioned only showed
>> up in a few posts, where linking was an issue. I have also gone
>> through the setup.py script and explicitly defined all the library
>> paths.
>>
>> My questions:
>>
>> - Has anyone ever ported Python 2* modules/libs to Python 3 that rely
>> on swig, and are there some changes in the C-API/swig I need to be
>> looking for to make this port successful?
>>
>> - Does anyone have any advice/insght about how I can troubleshoot,
>> diagnose, and resolve this issue?
>>
>> Thanks in advance,
>>
>> -- Adam
>>
>>
>>
>> 0:pylibpcap-0.6.2$ python3
>> Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.>>>
>> import pcap,py
>>
>> Traceback (most recent call last):
>> File "", line 1, in
>> File
>> "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
>> line 25, in
>> _pcap = swig_import_helper()
>> File
>> "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
>> line 21, in swig_import_helper
>> _mod = imp.load_module('_pcap', fp, pathname, description)
>> ImportError:
>> dlopen(/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so,
>> 2): Symbol not found: _PyCObject_FromVoidPtr
>> Referenced from:
>> /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
>> Expected in: flat namespace
>> in
>> /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
>>
>>
>>
>> >>> ^D- Hide quoted text -
>>
>> - Show quoted text -
>
> This is a change in the C-API in 3.2. See http://bugs.python.org/issue5630
>
> casevh
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Porting Python2 C-API/Swig based modules to Python 3
Link to pastebin: http://pastebin.com/102fhkgp
On Wed, Feb 23, 2011 at 6:18 PM, Adam Pridgen
wrote:
> Thanks for the help. That information got me started in the right
> direction. I put my notes up on paste bin for others to use in the
> future.
>
> For the future reference of others, I have included the pertinent
> details below. My process for identifying the various issues was
> compiling and running the project to see what was still broken after
> each fix. I know this is not the most efficient way, but it worked
> for this pylibcap since the code base was pretty small. I don't think
> everything is completely fixed, but its a step in the right direction.
>
> The initial hurdle I encountered was swig. I had to modify swig's
> code output, because it was not using the PyCapsule_* API yet. I
> found someones solution here: http://paste.pocoo.org/show/253494/.
> After applying that fix, I found that the pcap_interface.c and pcap.i
> files in the project were using some deprecated API calls. So I
> converted the following APIs:
>
> PyLong_AS_INT ==> PyLong_AsLong
> PyInt_* ==> PyLong_*
> PyString_* ==> PyBytes_* (initialily changed to PyUnicode_* but that
> was wrong, because I needed the Raw Bytes)
>
> I also had to change s# to y#. Before, s# stood for string and #, but
> it now means unicode and number.
>
>
>
> Hopefully these details will be helpful to others in the future.
> Thanks again for the nudge,
>
> -- Adam.
>
> On Wed, Feb 23, 2011 at 11:58 AM, casevh wrote:
>> On Feb 23, 8:54 am, Adam Pridgen
>> wrote:
>>> Hello,
>>>
>>> I am trying to get a compiled module to work with Python3. The code I
>>> am compiling was originally intended to be used in a Python 2.*
>>> environment, but I updated all the Python 2.* elements and the swig
>>> commands used by the setup.py script. I got the library to
>>> successfully compile, but I am getting the following errors on import
>>> (below). I am not sure how to trouble shoot this problem, and the
>>> fact that only one symbol (_PyCObject_FromVoidPtr) is missing is
>>> disconcerting. I Googled some, but the symbol mentioned only showed
>>> up in a few posts, where linking was an issue. I have also gone
>>> through the setup.py script and explicitly defined all the library
>>> paths.
>>>
>>> My questions:
>>>
>>> - Has anyone ever ported Python 2* modules/libs to Python 3 that rely
>>> on swig, and are there some changes in the C-API/swig I need to be
>>> looking for to make this port successful?
>>>
>>> - Does anyone have any advice/insght about how I can troubleshoot,
>>> diagnose, and resolve this issue?
>>>
>>> Thanks in advance,
>>>
>>> -- Adam
>>>
>>>
>>>
>>> 0:pylibpcap-0.6.2$ python3
>>> Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
>>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>>> Type "help", "copyright", "credits" or "license" for more information.>>>
>>> import pcap,py
>>>
>>> Traceback (most recent call last):
>>> File "", line 1, in
>>> File
>>> "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
>>> line 25, in
>>> _pcap = swig_import_helper()
>>> File
>>> "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
>>> line 21, in swig_import_helper
>>> _mod = imp.load_module('_pcap', fp, pathname, description)
>>> ImportError:
>>> dlopen(/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so,
>>> 2): Symbol not found: _PyCObject_FromVoidPtr
>>> Referenced from:
>>> /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
>>> Expected in: flat namespace
>>> in
>>> /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
>>>
>>>
>>>
>>> >>> ^D- Hide quoted text -
>>>
>>> - Show quoted text -
>>
>> This is a change in the C-API in 3.2. See http://bugs.python.org/issue5630
>>
>> casevh
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: lxml
On 24/02/2011 00:10, Colin J. Williams wrote: Could someone please let me know whether lxml is available for Windows XP?. If so, is it available for Python 2.7? The latest stable release is here: http://pypi.python.org/pypi/lxml/2.2.8 -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
On Wed, 23 Feb 2011 23:54:22 -, Ben Finney wrote: grobs456 writes: I realize I can double click on a .py file and Windows treats it as an executable but the command prompt pops in and out so quickly that I can't see the results of my script. Don't do that, then :-) You can open a persistent terminal window, and run the program from the command line. Being Windows, this is hidden in a filing cabinet in the basement behind a door marked "Beware of the Leopard." Or rather, somewhere in the "Accessories" item in the Start menu. I forget exactly what it's called (and I'm not using Windows at the moment, so I can't check) because I always drag a link to the QuickStart tray whenever I first use a Windows machine. Python is installed at: C:\Python27 and I ran: set path=%path%;C:\python27 #do i have to run the above each time I open up a session? You should be able to set that value persistently. (I don't wash Windows, so I don't know what you need to do there.) Control Panel => System, on the "Advanced" tab there's a button labelled "Environment Variables" (or something like that). You'll want to edit the current PATH setting (in the lower half of the dialog) to add ";C:\python27" to the end. Obvious, isn't it :-/ -- Rhodri James *-* Wildebeest Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: Running Scripts vs Interactive mode
On 2/23/2011 5:22 PM, grobs456 wrote: Hi, I am trying to work through the tutorial at: http://docs.python.org/tutorial/ The issue I am facing is with regards to the discussion about "Invoking the Interpreter" and "Executable Python Scripts". It was rather hazy in my opinion. An alternative to the other solutions given is to edit in an IDLE editor window and run with F5. Results are printed in the Shell window and when the script is finished, you can interactively examine any object created. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: QWebView: notify python when I clicked on a certain web eleemnt
Damjan,
Thanks once more for your help.
You pointed me in the right direction.
On 02/24/2011 12:25 AM, Ian Kelly wrote:
> On Wed, Feb 23, 2011 at 4:02 PM, Gelonida wrote:
>>> so make a Python object with some methods you'd wish to call, then
>>> evaluate some small JS that will bind the click event on your element to
>>> the method of the Python object.
>>>
>>
>> OK, but how do I make sure, that I don't override an existing binding to
>> the element's click() event?
>
> If the element has an existing binding, then create a new JS function
> that calls both the Python method and the original event handler, and
> bind that function to the event.
Not sure if it can be done easier, but what I roughly do now is:
top_frame = page.mainFrame()
my_qt_object = MyQtObject()
top_frame.addToJavaScriptWindowObject("pyObj", my_qt_object)
field = top_frame.findFirstElement(path)
js_inject_after = """{
var that = this;
var prev_action = this.%(evt)s;
if (prev_action == null){
prev_action = function(){};
}
this.%(evt)s = function(){
prev_action();
%(action)s;
};
}"""
prms = dict(
evt="onclick",
action=""" pyObj.action(that.id+ "clicked"); """
)
field.evaluateJavaScript(js_inject_after % prms)
--
http://mail.python.org/mailman/listinfo/python-list
Re: QWebView: notify python when I clicked on a certain web eleemnt
Thanks Ian, Pls see my other reply. Is that what you meant? On 02/24/2011 12:25 AM, Ian Kelly wrote: > On Wed, Feb 23, 2011 at 4:02 PM, Gelonida wrote: >>> so make a Python object with some methods you'd wish to call, then >>> evaluate some small JS that will bind the click event on your element to >>> the method of the Python object. >>> >> >> OK, but how do I make sure, that I don't override an existing binding to >> the element's click() event? > > If the element has an existing binding, then create a new JS function > that calls both the Python method and the original event handler, and > bind that function to the event. -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess pipe question
Rita wrote in news:AANLkTi=88dcpm_kqrs2g620obsnxz0majubfwpeme...@mail.gmail.com in gmane.comp.python.general: [Top post relocated] > On Tue, Feb 22, 2011 at 7:57 PM, Rob Williscroft > wrote: > >> Rita wrote in >> news:AANLkTi=w95gxosc1tkt2bntgjqys1cbmdnojhokq4...@mail.gmail.com in >> gmane.comp.python.general: >> >> > >> > When using wait() it works a bit better but not consistent >> > def run(cmd): >> > p=subprocess.Popen(cmd,stdout=subprocess.PIPE) >> > rc=p.wait() >> > print rc >> > return p.stdout >> > >> >> > When the output of cmd is a small ascii file it works perfectly >> > fine, but when the file is large (more than 2MB) the process just >> > waits for ever (I am guessing its blocking?). >> >> Your OS has supplied a pipe buffer of 2MB, its full and the prossess >> is waiting until you read something from the pipe (i.e. >> p.stdout.read()). >> >> >When I use the communicate call >> > it works perfectly but my process is consuming way too much memory. >> > >> > Is there a better way to get my return code consistently >> > efficiently and not take up so much memory? >> >> If you don't need the processes output then don't use the PIPE >> argument. If you do you will need to read from p.stdout until the >> process is complete, then get the return code: >> >> while True: >>buf = p.stdout.read( 1024 ) >># do somthing with buf >>if len( buf ) < 1024: >>break >> >> rc = p.wait() > > For extra points is there a way to speed up the p.stdout.read(bufsize) > ? Its unlikely that is the problem, most likely you are reading all of the 2MB OS pipe buffer and then having to wait for the programme you are calling to produce some more output. Before trying to speed things up be sure to test with real work being done to the output you recieve. If its still slow and you have multiple cores/processors then you will need to put you read loop in a thread to speed it up. -- Rob. -- http://mail.python.org/mailman/listinfo/python-list
pattern matching
if I have a string such as '01/12/2011' and i want
to reformat it as '20110112', how do i pull out the components
of the string and reformat them into a DDMM format?
I have:
import re
test = re.compile('\d\d\/')
f = open('test.html') # This file contains the html dates
for line in f:
if test.search(line):
# I need to pull the date components here
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is setdefaultencoding bad?
On Wed, 23 Feb 2011 04:14:29 -0800, Chris Rebert wrote: >> Ok, but that the interface handles UTF-8 strings >> are still ok? The defaultencoding is still ascii. > > Yes, that's fine. UTF-8 is an excellent encoding choice, and > encoding/decoding should always be done explicitly in Python, so the > "default encoding" ideally ought to never come into play (and indeed, > Python 3 does away with bug-prone implicit encoding/decoding entirely > FWICT). On Unix, you have to go out of your way to avoid the use of implicit encoding/decoding with the "filesystem" encoding. This is because Unix extensively uses byte strings with no associated encoding, but Python 3 tries to use Unicode for everything. 3.0 was essentially unusable as a Unix scripting language for this reason, as argv and environ were converted to Unicode, with no possibility of recovering from unconvertible sequences. 3.1 added the surrogate-escape mechanism which allows recovery of the original byte sequences, albeit with some effort (i.e. you had to explicitly decode os.environ and sys.argv). 3.2 adds os.environb (bytes version of os.environ), but it appears that sys.argv still has to be encoded manually. It also provides os.fsencode() and os.fsdecode() to simplify the conversion. Most functions accept bytes arguments, most either return bytes when passed bytes or (if the function accepts no arguments) has a bytes equivalent. But variables tend to be Unicode strings with no bytes version (os.environb is the exception rather than the rule), and some functions have no bytes equivalent (e.g. os.ctermid(), os.uname(), os.ttyname(); fortunately it's rather unlikely that the result from any of these functions will contain non-ASCII characters). -- http://mail.python.org/mailman/listinfo/python-list
Re: Re: Running Scripts vs Interactive mode
On 01/-10/-28163 02:59 PM, Ben Finney wrote: grobs456 writes: I realize I can double click on a .py file and Windows treats it as an executable but the command prompt pops in and out so quickly that I can't see the results of my script. Don't do that, then :-) You can open a persistent terminal window, and run the program from the command line. You get at least three other benefits: You can specify arguments (available in sys.argv) for each run, to test the program in various ways. And you can see earlier runs on the screen, and scroll back an arbitrary (configurable) amount to see what changed. And you can copy/paste that output elsewhere, like to an email if you have a question. and I ran: set path=%path%;C:\python27 #do i have to run the above each time I open up a session? The value will survive within the one command window, and any other command windows you start from it. But if you want that value to be available on all (new) command windows, you set it in the Control Panel. It's been a while since I ran Windows, but it was someplace like System->EnvironmentVariables) Once you do that, it'll not only be available in any new DOS sessions, and any application that's run from Explorer, but also will survive a restart. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: pattern matching
In article ,
monkeys paw wrote:
> if I have a string such as '01/12/2011' and i want
> to reformat it as '20110112', how do i pull out the components
> of the string and reformat them into a DDMM format?
>
> I have:
>
> import re
>
> test = re.compile('\d\d\/')
> f = open('test.html') # This file contains the html dates
> for line in f:
> if test.search(line):
> # I need to pull the date components here
My first thought is that any attempt to parse HTML by using regex is
doomed to failure. HTML is meant to be parsed by an HTML parser.
Python gives you several to pick from; the best that I know of is the
third-party lxml package (http://lxml.de/).
My second thought is that my first thought was correct.
--
http://mail.python.org/mailman/listinfo/python-list
Re: pattern matching
On Wed, 23 Feb 2011 21:11:53 -0500, monkeys paw wrote:
> if I have a string such as '01/12/2011' and i want to reformat
> it as '20110112', how do i pull out the components of the string and
> reformat them into a DDMM format?
data = '01/12/2011'
# Throw away tags.
data = data[4:-5]
# Separate components.
day, month, year = data.split('/')
# Recombine.
print(year + month + day)
No need for the sledgehammer of regexes for cracking this peanut.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: pattern matching
On Wed, Feb 23, 2011 at 6:37 PM, Steven D'Aprano
wrote:
> On Wed, 23 Feb 2011 21:11:53 -0500, monkeys paw wrote:
>> if I have a string such as '01/12/2011' and i want to reformat
>> it as '20110112', how do i pull out the components of the string and
>> reformat them into a DDMM format?
>
> data = '01/12/2011'
> # Throw away tags.
> data = data[4:-5]
> # Separate components.
> day, month, year = data.split('/')
> # Recombine.
> print(year + month + day)
>
>
> No need for the sledgehammer of regexes for cracking this peanut.
Agreed. But "Just 'Cause"(tm), and in order to get in some regex practice:
from re import compile
regex = compile("(\d\d)/(\d\d)/(\d{4})")
for match in regex.finditer(data):
day, month, year = match.groups()
print(year + month + day)
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: pattern matching
In article ,
Chris Rebert wrote:
> regex = compile("(\d\d)/(\d\d)/(\d{4})")
I would probably write that as either
r"(\d{2})/(\d{2})/(\d{4})"
or (somewhat less likely)
r"(\d\d)/(\d\d)/(\d\d\d\d)"
Keeping to one consistent style makes it a little easier to read. Also,
don't forget the leading `r` to get raw strings. I've long since given
up trying to remember the exact rules of what needs to get escaped and
what doesn't. If it's a regex, I just automatically make it a raw
string.
Also, don't overlook the re.VERBOSE flag. With it, you can write
positively outrageous expressions which are still quite readable. With
it, you could write this regex as:
r" (\d{2}) / (\d{2}) / (\d{4}) "
which takes up only slightly more space, but makes it a whole lot easier
to scan by eye.
I'm still going to stand by my previous statement, however. If you're
trying to parse HTML, use an HTML parser. Using a regex like this is
perfectly fine for parsing the CDATA text inside the HTML element,
but pattern matching the HTML markup itself is madness.
--
http://mail.python.org/mailman/listinfo/python-list
Re: pattern matching
if I have a string such as '01/12/2011' and i want
to reformat it as '20110112', how do i pull out the components
of the string and reformat them into a DDMM format?
I have:
import re
test = re.compile('dd/')
f = open('test.html') # This file contains the html dates
for line in f:
if test.search(line):
# I need to pull the date components here
I am no python guru but you could use beautifulsoup to parse html as its
much easier
some untested pseudocode below. adapt to your needs.
from BeautifulSoup import BeautifulSoup
#read html data or whatever source
html_data = open('/yourwebsite/page.html','r').read()
#Create the soup object from the HTML data
soup = new BeautifulSoup(html_data)
someData = soup.find('td',name='someTable')
#Find the proper tag see beautifulsoup docs
value = someData.attrs[2][1] # the value of 3rd attrib of the tag , just
an example
##end
now when you have the date in some str format the next thing is your date
conversion. For this
re fer to dateutil parse http://labix.org/python-dateutil
hope it help.
posted via Grepler.com -- poster is authenticated.
begin 644
end
--
http://mail.python.org/mailman/listinfo/python-list
Re: Re: Securing files
On Wed, Feb 23, 2011 at 11:48 AM, Timothy W. Grove wrote: Hello Folks, In a python application that I'm developing I've been asked to add security to databases that the program might create and access; the database is to be password protected by its creator. The application uses an SQLite database, which could be changed for another back-end if that would offer better security, but I would still like to use an embeddable database file. The problem isn't so much the database itself, as I can think of a number of way to encrypt the data it contains, but some of the data is simply names of image and video files contained elsewhere in the file-system. Is there anyway to prevent a user from simply opening up the file-system from outside of the application and viewing the files? One way that I can think of would be to encode the image/video files as BLOBS and store them in the database itself, but apart from that option, can anyone suggest other ways? I'm currently working with python2.7 under Windows7, but I'm hoping to extend the application to Linux and Mac also. Thank you for your help. I don't know much about your system, but just as a hunch I'd say that if this is a major requirement you're screwed. The old maxim: there is no cryptographic solution to the problem in which the attacker and intended recipient are the same person. Geremy Condra Create a Group or User permission object in Windows where only your application is the member of the group and assign file access to ONLY this User/Group. This way regular users apart from administrator cannot access the folder/files. Thats one way of doing it if i understand you correctly. posted via Grepler.com -- poster is authenticated. begin 644 end -- http://mail.python.org/mailman/listinfo/python-list
Re: Python for embedded systems?
Paulito writes: > "Is Python 'mature' enough to be considered the primary language for > embedded systems? Is the performance there for real-time applications > (eg avionics, real-time control systems) or is it still more suitable > "...as a glue language, used to combine components written in C++" ?" IMO it's not a matter of maturity, but of design features. If you mean hard real time, Python really is not suited to that type of application, which has rather specialized requirements. Python is surely used in some soft realtime applications since lots of embedded devices these days are fairly standard linux boxes under the skin. > And further: > "Has anyone tried to shorten development time when porting code to a > new embedded hardware platform, by trying to convert legacy code (C/C+ > +/Ada) to Python?" That would be a re-implementation rather than a port. Closest thing I can think of is Ericsson switching from C++ to Erlang in its phone switches. > I'm currently thinking that Python isn't there yet but certainly would > like to hear any feedback. Do you have a specific application in mind? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python for embedded systems?
On Feb 23, 6:53 pm, Paulito wrote: > Apologies if this has been asked; I haven't yet Googled the archives. > > From a brief email conversation, Guido pointed me to this newsgroup to > ask the following questions: > > "Is Python 'mature' enough to be considered the primary language for > embedded systems? Is the performance there for real-time applications > (eg avionics, real-time control systems) or is it still more suitable > "...as a glue language, used to combine components written in C++" ?" > > And further: > "Has anyone tried to shorten development time when porting code to a > new embedded hardware platform, by trying to convert legacy code (C/C+ > +/Ada) to Python?" > > I'm currently thinking that Python isn't there yet but certainly would > like to hear any feedback. > > Your input is greatly appreciated; thanks! > - Paulito Python is probably not the best choice for embedded systems because it lacks fine control over hardware, something that C and C++ have in bunches. Also, it doesn't perform well enough to be considered for situations where resources are at a premium (think microcontrollers). Could you develop in Python for a platform like Android or iPhone? Yeah, they have the space, memory, and CPU to run Python stuff. But on weaker CPUs and less memory, Python would be a poor choice. It's not a matter of language maturity, Python is very mature, it's a matter of design. Python is a high-level, garbage-collected, interpreted language, and that's not the ideal type of language for embedded systems. Rafe -- http://mail.python.org/mailman/listinfo/python-list
