Re: AssertionError - help me to solve this in a programme with Queue
> hi > It did work that way > > File "queue.py", line 15 > threading.Thread__init__.(self) > ^ > SyntaxError: invalid syntax You've wrote right first time "threading.Thread.__init__(slef)", but misprinted in your code "def _int_(self,q)" instead of "def __init__ (self,q)". -- http://mail.python.org/mailman/listinfo/python-list
[announcement] - python graph library
Hello Folks, I recently started working on a graph-algorithms library in Python. What makes this one different from the other couple of libs that are available is a heavy influence from the C++ Boost Graph Library. There are IMO a lot of good ideas there, so I am attempting to translate the spirit of it into Python without loosing the Pythonness :). There is no official code release so far, but I have been blogging ideas and code snippets here: http://pythonzweb.blogspot.com/. Your comments are most welcome (you can leave them right on the blog).I also wanted to tap your opinion on naming this thing. There is (duh) already PyGL and PGL names associated with python (but not graph libs) floating around, but as they are not so well-known as far as I can tell I do not mind taking them on and stealing the name. What do you guys think? -- http://mail.python.org/mailman/listinfo/python-list
unicode converting
there are a few questions i can find answer in manual: 1. how to define which is internal encoding of python unicode strings (UTF-8, UTF-16 ...) 2. how to convert string to UCS-2 (Python 2.2.3 on freebsd4) -- Best regards, Maxim -- http://mail.python.org/mailman/listinfo/python-list
Re: unicode converting
Diez B. Roggisch wrote:
Maxim Kasimov wrote:
there are a few questions i can find answer in manual:
1. how to define which is internal encoding of python unicode strings
(UTF-8, UTF-16 ...)
It shouldn't be your concern - but you can specify it using " ./configure
--enable-unicode=ucs2" or --enable-unicode=ucs4. You can't set it to utf-8
or utf-16.
is that means that python internal unicode format is ucs2 or ucs4?
i'm concerning with the qustion because i need to send data to external
application in ucs2 encoding
2. how to convert string to UCS-2
s = ... # some ucs-2 string
s.decode("utf-16")
not _from_ ucs2, but _to_ ucs2, for example:
s = ... # some utf-16 string
d = encode_to_ucs2(s)
might give you the right results for most cases:
http://mail.python.org/pipermail/python-dev/2002-May/024193.html
--
Best regards,
Maxim
--
http://mail.python.org/mailman/listinfo/python-list
Re: unicode converting
Christos TZOTZIOY Georgiou wrote:
If unicode_data references your unicode data, all you have to send is:
unicode_data.encode('utf-16') # maybe utf-16be for network order
is utf-16 string the same ucs-2? my question is how to get string encoded as
UCS-2
--
Best regards,
Maxim
--
http://mail.python.org/mailman/listinfo/python-list
Re: unicode converting
Serge Orlov wrote: utf-16 is basically a superset of ucs-2. See here for more detail: http://www.azillionmonkeys.com/qed/unicode.html If you ensure that ord() of each output character is < 0x1 you'll get valid ucs-2 output if you use utf-16 encoding. If you build python with --enable-unicode=ucs2 no character can be >= 0x1 so you don't have to check. thank you very match! that's exactly what i need -- Best regards, Maxim -- http://mail.python.org/mailman/listinfo/python-list
libxml2/xpath
I am trying to do some xpath on
http://fluidobjects.com/doc.xhtml
but cannot get past 'point A' (that is, I am totally stuck):
>> import libxml2
>> mydoc = libxml2.parseDoc(text)
>> mydoc.xpathEval('/html')
>> []
this returns an empty resultlist, which just seems plain wrong. Can anyone
throw a suggestion to the stupid?
thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: libxml2/xpath
I do not believe it is... You can see the doc by clicking on the link.
Does it have to be?
thanks,
m
Martijn Faassen wrote:
Maxim Khesin wrote:
I am trying to do some xpath on
http://fluidobjects.com/doc.xhtml
but cannot get past 'point A' (that is, I am totally stuck):
import libxml2
mydoc = libxml2.parseDoc(text)
mydoc.xpathEval('/html')
[]
this returns an empty resultlist, which just seems plain wrong. Can
anyone
throw a suggestion to the stupid?
Is the html element in a namespace?
Regards,
Martijn
--
http://mail.python.org/mailman/listinfo/python-list
portable text user interface
Hi. Are there widely used and recommended Python libraries that will let me makes a portable text user interface? -- Best regards, Maxim -- http://mail.python.org/mailman/listinfo/python-list
Re: portable text user interface
Miki Tebeka wrote: Hello Fuzzyman, Are there widely used and recommended Python libraries that will let me makes a portable text user interface? If you just need a text-like interface you can use Tkinter. See (shameless plug) http://developer.berlios.de/projects/bcd/ and http://developer.berlios.de/dbimage.php?id=1112 for example. Your project looks very interesting. It would be better if it displayed an error message if it can't find the '_bcdrc' file. If you run it from windoze it just appears and disapears. How about allowing the '_bcdrc' file to be in the same directory as the script as well. Cool. A user! Thanks for the comments. I'll add searching next to the application on win32 platforms. Any chance of you releasing the Tkinter text interface as a separate library, with a less restrictive license ? It looks very good - but I can't use it in my projects if it is GPL. It's just standard Tkinter Listbox with key bindings and it's under 160 lines of code. Nothing special here. Bye. -- Miki Tebeka <[EMAIL PROTECTED]> http://tebeka.bizhat.com The only difference between children and adults is the price of the toys i have not used Tcl/Tk before, and don't know how to solve the problem - after i've made config file (.bcdrc) and run the script on FreeBSD 4, i've got the message: Traceback (most recent call last): File "./bcd.py", line 177, in ? root = Tk() # Main window File "/usr/local/lib/python2.2/lib-tk/Tkinter.py", line 1511, in __init__ self.tk = _tkinter.create(screenName, baseName, className) TclError: no display name and no $DISPLAY environment variable -- Best regards, Maxim -- http://mail.python.org/mailman/listinfo/python-list
Re: portable text user interface
Steve Holden wrote: Maxim Kasimov wrote: Miki Tebeka wrote: Hello Fuzzyman, Are there widely used and recommended Python libraries that will let me makes a portable text user interface? If you just need a text-like interface you can use Tkinter. See (shameless plug) http://developer.berlios.de/projects/bcd/ and http://developer.berlios.de/dbimage.php?id=1112 for example. Your project looks very interesting. It would be better if it displayed an error message if it can't find the '_bcdrc' file. If you run it from windoze it just appears and disapears. How about allowing the '_bcdrc' file to be in the same directory as the script as well. Cool. A user! Thanks for the comments. I'll add searching next to the application on win32 platforms. Any chance of you releasing the Tkinter text interface as a separate library, with a less restrictive license ? It looks very good - but I can't use it in my projects if it is GPL. It's just standard Tkinter Listbox with key bindings and it's under 160 lines of code. Nothing special here. Bye. -- Miki Tebeka <[EMAIL PROTECTED]> http://tebeka.bizhat.com The only difference between children and adults is the price of the toys i have not used Tcl/Tk before, and don't know how to solve the problem - after i've made config file (.bcdrc) and run the script on FreeBSD 4, i've got the message: Traceback (most recent call last): File "./bcd.py", line 177, in ? root = Tk() # Main window File "/usr/local/lib/python2.2/lib-tk/Tkinter.py", line 1511, in __init__ self.tk = _tkinter.create(screenName, baseName, className) TclError: no display name and no $DISPLAY environment variable This is a problem with window creation. I presume it's occurring because you aren't running the program in a windowed environment - if you've logged in through a graphical screen such as xdm and you are running the program in a window on a desktop the DISPLAY environment variable is normally set up for you. Perhaps you are just telnetting in from a remote system? regards Steve yes i'm telneting (sshing), that is the reason why i'm looking for libs for making text interfaces. i know there is a project named "anakonda" - red hat linux installer, but it is uses specific C libs. i can use only python libs. -- Best regards, Maxim -- http://mail.python.org/mailman/listinfo/python-list
Re: portable text user interface
Grant Edwards wrote: On 2004-12-30, Maxim Kasimov <[EMAIL PROTECTED]> wrote: yes i'm telneting (sshing), that is the reason why i'm looking for libs for making text interfaces. i know there is a project named "anakonda" - red hat linux installer, but it is uses specific C libs. i can use only python libs. By "only python libs" do you mean only the stuff that's included in the vanilla CPython distro? If that's the case, then curses is pretty much the only choice. Last time I looked, it didn't work on Windows. I've used the snack module (which is what anaconda uses), and it's very handy for simple text-based UIs. at ports (FreeBSD) i've found this: Port: snack-2.2.7 Path: /usr/ports/audio/snack Info: A sound toolkit for scripting languages and at http://rpmfind.net i've found: sound extension for Tcl/Tk and Python so if it's about http://www.speech.kth.se/snack/ not sure is it what i need but will try and thanks for help. -- Best regards, Maxim -- http://mail.python.org/mailman/listinfo/python-list
Re: portable text user interface
Grant Edwards wrote: On 2004-12-30, Maxim Kasimov <[EMAIL PROTECTED]> wrote: yes i'm telneting (sshing), that is the reason why i'm looking for libs for making text interfaces. i know there is a project named "anakonda" - red hat linux installer, but it is uses specific C libs. i can use only python libs. By "only python libs" do you mean only the stuff that's included in the vanilla CPython distro? If that's the case, then curses is pretty much the only choice. Last time I looked, it didn't work on Windows. I've used the snack module (which is what anaconda uses), and it's very handy for simple text-based UIs. at ports (FreeBSD) i've found this: Port: snack-2.2.7 Path: /usr/ports/audio/snack Info: A sound toolkit for scripting languages Unfortunately there are two modules with the same name. The one you found is a sound library. The other one is the Python wrapped version of the newt library. I've no idea why the Python newt module is called "snack". The new source code comes with the Python "snack" module wrapper. The most recent version I have handy is available at ftp://ftp.visi.com/users/grante/stuff/newt-0.50.tar.gz There appear to be more recent versions available: http://rpmfind.net/linux/RPM/fedora/3/i386/newt-0.51.6-5.i386.html http://linux.maruhn.com/sec/newt.html http://www.freshports.org/devel/newt There's a rather outdated tutorial on using Newt v0.30 from C: http://www.oksid.ch/gnewt/tutorial.html Newt requires the s-lang library: http://www.s-lang.org/ that is exactly i need. i'll try it. thanks. -- Best regards, Maxim -- http://mail.python.org/mailman/listinfo/python-list
Re: Archives and magic bytes
> Another thing, I work on linux (gentoo) and I would like to use the
"file" command to retrieve informations about type of file instead of
using extensions, do you think this can be done?
this is trivial:
>>> import os
>>> os.popen("file /etc/passwd").read()
'/etc/passwd: ASCII text\n'
--
http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
Mage wrote: praba kar wrote: Dear All, In Python what is equivalent to goto statement You shouldn't use goto in high-level languages. it would be quite useful for debuging porposes -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
Simon Brunning wrote: On 4/20/05, Maxim Kasimov <[EMAIL PROTECTED]> wrote: it would be quite useful for debuging porposes How does goto help you to remove bugs? I can certainly see how it helps you put them in in the first place... if you need to comment a couple of code (and then uncomment ), what are you doing then? -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
Robert Kern wrote: Maxim Kasimov wrote: Simon Brunning wrote: On 4/20/05, Maxim Kasimov <[EMAIL PROTECTED]> wrote: it would be quite useful for debuging porposes How does goto help you to remove bugs? I can certainly see how it helps you put them in in the first place... if you need to comment a couple of code (and then uncomment ), what are you doing then? Use comments? WOW, just greate! ... but i'd like to relax at some more interesting way than to comment each of rows -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
Use multi-line string literals. ''' it will not help if there is another ''' or/and """ inside of code block This whole 'code' is "commented out", and you can use every type of """quote""" except three singles. ''' Or, if you really like the spirit of goto, use "if 0:". ... and add tabs to each string Reinhold -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
Torsten Bronger wrote: HallÃchen! Maxim Kasimov <[EMAIL PROTECTED]> writes: [...] WOW, just greate! ... but i'd like to relax at some more interesting way than to comment each of rows but what if i just can't to do this becouse i'm working thrue ssh, and have to use only installed editors (such as vi) -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
Peter Hansen wrote: Maxim Kasimov wrote: Torsten Bronger wrote: HallÃchen! Maxim Kasimov <[EMAIL PROTECTED]> writes: WOW, just greate! ... but i'd like to relax at some more interesting way than to comment each of rows but what if i just can't to do this becouse i'm working thrue ssh, and have to use only installed editors (such as vi) Surely you use more than one-character indents? If that's so, you can just insert the "if 0:" using, say, two characters indentation instead of the usual four. Then the following four-character-indented code is removed. Alternatively, use ''' and ''' surrounding the code block and it will be ignored. f..., i don't requesting that "goto" was available in next versions of python, but i'm saying if it will be so, it will be easy and quickly _debug_ some skripts, _not only_ for commenting -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
Christos TZOTZIOY Georgiou wrote:
On Wed, 20 Apr 2005 16:13:32 +0300, rumours say that Maxim Kasimov
<[EMAIL PROTECTED]> might have written:
but what if i just can't to do this becouse i'm working thrue ssh, and have to use only installed editors (such as vi)
If you use plain vi (not vim) and you want to comment e.g. 5 lines of
code, go to the first of these five and:
- if autoindent type:
5>>O^Dif 0:{ESC}
^D above is Ctrl-D, {ESC} is your Escape key
- if noautoindent type:
YP^Cif 0:{ESC}j5>>
^C above is literal ^, literal C, *NOT* Ctrl-C; {ESC} is your Escape key
To control autoindent, you can type:
:se ai
or
:se noai
If you need more help, I would gladly send you the output of `man vi'
from a non-GNU Unix. I can also send you the output of `man vim' from a
GNU system.
is it wrong to debug python script using python, but not some magic
commands found somewhere in man ?
--
Best regards,
Maxim Kasimov
mailto: [EMAIL PROTECTED]
--
http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
by the way, "goto" statement will be useful for writing more powerful obfuscators -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: Troll? was: Re: goto statement
André Roberge wrote: Maxim Kasimov wrote: by the way, "goto" statement will be useful for writing more powerful obfuscators Let me get that clear: you want a goto to help with debugging. And you want to obfuscate your code even more? !? Perhaps you need to write in Perl, or some other similar language. Writing in Python is for those that seek clarity (not obfuscation) and less bugs. Which is why a goto statement should definitely never be part of Python! André so insulting to me - you asking i'm a troll, only becose i'm saing that goto maybe sometimes usefull. -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
1. comment for debug It can be used in the same way, as the comments for debugging are used, but it will be easier than to use """ or ''', or using features of text-editors, when it is necessary to comment piece of code which already contains ''' or/and """ strings already, or there is another #-comments. Using goto, you do not need to edit a code, which is unfamiliar to you. 2. obfuscators goto can be used in the same way, as many of java-obfuscators do Speaking in other words: 1) goto exempts from necessity to install new software (it is critical for remote working, for example, installing X11 may be impossible at all) 2) enables to make new, better software (better obfuscators) -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
Sergei Organov wrote: Maxim Kasimov <[EMAIL PROTECTED]> writes: 1. comment for debug It can be used in the same way, as the comments for debugging are used, but it will be easier than to use """ or ''', or using features of text-editors, when it is necessary to comment piece of code which already contains ''' or/and """ strings already, or there is another #-comments. Using goto, you do not need to edit a code, which is unfamiliar to you. and then you end up with situation when another goto jumps directly into the body of the code you've just "commented" with your goto. Happy debugging! BTW, don't you want "comefrom" statement to be added to the language to make debugging even more fun? if you can't control what you do - it is you private problem, not mine. Is't it? -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: goto statement
pythonUser_07 wrote: Hi, Have you tried the triple quote comment technique? I am assuming you want to skip some code for the time being. Here is an example print "hello world" ''' COMMENT OUT FOR NOW someFunction() someOtherFunction() ''' print "goodbye world" This means that you have only two locations to remove the blocked out code. This is identical to having to remove the goto statement and the marker. Hope that helps. how do use this here: print "hello world" ... ... ... sql = ''' some long query ''' ... ... ... sql = """ another query """ ... ... ... print "goodbye world" -- Best regards, Maxim Kasimov mailto: [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: help needed :-pgdb givig error
Please try to check out permisions on the library. Perhaps there is no permissions for reading the library with uid of web-server "Ram" <[EMAIL PROTECTED]> ???/ ? ?: news:[EMAIL PROTECTED] Dear All I am very new to python . i would appreciate any help from you all on this problem which i am facing. I am trying to connect to postgres from python.for this i am using something like " import pgdb". i am able to connect to postgres and fetch data , if i execute the python file directly in unix prompt. However when i try to do thsi through broeser iam getting the following error.. Traceback (most recent call last): File "/opt/tools/cvs/htdocs/viewcvs/cgi/test.cgi", line 5, in ? import sys,string,os,pgdb File "/usr/local/lib/python2.3/site-packages/pgdb.py", line 62, in ? from _pg import * ImportError: ld.so.1: /usr/local/bin/python: fatal: libpq.so.3: open failed: No such file or directory premature end of script headers... I have python2.3 and 2.4 installed , of which pgdb.py and pg.py is installed in /usr/local/lib/python2.3/site-packages . Please help me out how to solve this when i give $python it is going to python 2.3.3. I also have the shared library _pg.so in the same directory. Luv Ram -- http://mail.python.org/mailman/listinfo/python-list
Re: python equivalent of php implode
i'm tying to run example, and then get a traceback. am i something missed?
mysql> create table tmp_tmp (id int not null auto_increment primary key, sd
varchar(255) not null default '', si int not null default 1);
>>> import MySQLdb
>>> db = MySQLdb.connect("localhost", "login", "password", "dbname")
>>> c = db.cursor()
>>> query_param = {
... 'sd' : 'somedata',
... 'si' : 2,
... }
>>> table = 'tmp_tmp'
>>> keys = query_param.keys()
>>> values = query_param.values()
>>> sql = "INSERT INTO %s (%s) values (%s)" % (table, ", ".join(keys), ",
".join(["?"] * len(keys)))
>>> c.execute(sql, values)
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/local/lib/python2.2/site-packages/MySQLdb/cursors.py", line 95,
in execute
return self._execute(query, args)
File "/usr/local/lib/python2.2/site-packages/MySQLdb/cursors.py", line
108, in _execute
self.errorhandler(self, ProgrammingError, m.args[0])
File "/usr/local/lib/python2.2/site-packages/MySQLdb/connections.py", line
33, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: not all arguments converted
"Mike Meyer" <[EMAIL PROTECTED]> ???/ ? ?:
news:[EMAIL PROTECTED]
> Jeff Epler <[EMAIL PROTECTED]> writes:
>
> > items = query_param.items()
> > keys = [item[0] for item in items]
> > values = [item[1] for item in items]
>
> Is there some reason not to do:
>
>keys = query_params.keys()
>values = query_params.values()
>
> That would seem to be a lot more obvious as to what was going on.
>
> Thanks,
> --
> Mike Meyer <[EMAIL PROTECTED]> http://www.mired.org/home/mwm/
> Independent WWW/Perforce/FreeBSD/Unix consultant, email for more
information.
--
http://mail.python.org/mailman/listinfo/python-list
article posting
Dear Python community members, I would be very appreciate you if somebody could give me the contacts of responsible person for posting articles on python.org resource. It could be chief editor`s or administrator`s name and e-mail, Thank you very much advance, Best Regards Maxim Lesnichenko Business Development Manager Redwerk phone: +1 (347) 329-1444 email: <mailto:ma...@redwerk.com> ma...@redwerk.com skype: redwerk.com web: <http://redwerk.com> http://redwerk.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Javascript is turning into Python?!
Paul Rubin wrote: > [Correction of earlier accidental crosspost] > > I hadn't seen this before. New Javascript 1.7 features: > > - Generators > - Iterators > - Array comprehensions > - Destructuring assignment > > Sounds like another language we know. > > http://developer.mozilla.org/en/docs/New_in_JavaScript_1.7 If you're talking about Mozilla Javascript, then most certanly yes. Mozilla team even plan to support Python in XUL. If you're talking about some standard (ECMAscript), then, well I haven't seen any progress there. -- http://mail.python.org/mailman/listinfo/python-list
psycopg2 features
Hello, clp and all people reading it! Recently I was porting my (small) app from psycopg to psycopg2 (they got me with this "2"). I read, that psycopg2 supports all features of psycopg and plus many more, however, when I started to port, I discovered, that psycopg2 lacks serialized connections and "commit on cursor" completely. Did I miss something or psycopg2 just isn't mature enough yet? BTW, is there any postgresql engine, that can fake nested transactions via savepoints feature? -- http://mail.python.org/mailman/listinfo/python-list
Re: What do you want in a new web framework?
Laurent Pointal wrote: > > Look at http://wiki.python.org/moin/WebFrameworks > > Do you *really* need to develop a *new* framework (maybe a scholl > exercise - it that case, KISS)? Isn't the main reason why there are so many of them is that all of them suck badly? -- http://mail.python.org/mailman/listinfo/python-list
Re: Minidom XML output - attributes in wrong order ?
> So it seems the dom module sorts the attributes alphabetically. Is there any > way I can prevent it from doing that ? What I want is to list them out in > the same order as they are added in the code... I don't know how to do what you ask, I'm just here to warn you that you shouldn't rely on the order of attributes in the document in any way, because this is implementation dependent. If you need to preserve order of some items -- use child elements instead of attributes. -- Regards, Maxim Sloyko -- http://mail.python.org/mailman/listinfo/python-list
Re: best way to get data into a new instance?
tobiah wrote: [snip] > class Employee: > > __init__(self, id): > self.id = id > > e = Employee(32445) > > ... later > > > e.firstname = 'foo' > e.lastname = 'bar' > > ... more as the information comes about. Personally, I think that this is not a good solution. How would the reader of your code guess what properties your object has? If you don't like to write too many assignments, you can at least do something like this: def __init__(self, id, **kw): self.id = id for name in ['first_name', 'last_name', 'age', 'salary', 'whatever']: self.__dict__[name] = kw.get(name, None) -- Maxim Sloyko -- http://mail.python.org/mailman/listinfo/python-list
Re: apache & mod_python
m.banaouas wrote: > Can i install and use "Apache 2.2.3" & "mod_python 3.2.10" (most recent > versions) without facing any known major issue ? Works fine for me. The only "known major issue" you can face is general non-threadsafety of Python interpreter. So, if you are using Apache MPM, you have to allow for it, or use framework that does it for you. -- http://mail.python.org/mailman/listinfo/python-list
Re: your opinion about psycopg vs pygresql
Martin P. Hellwig wrote: > However, given the choice, what in your opinion would be the reason why > someone would chose one over the other? Now I know this could easily get > into a flamewar, so if you comment (but please do so) I'll still > investigate that, since at this moment I don't even have a clue how they > differ and on what reason, why does PostgreSQL seem to favour pygresql > and Pythoneers psycopg? > > Thanks in advance. Well, my info can be a little out of date, since I researched this problem more than a year ago. AFAIK, psycopg still lacks prepared statements, which can be a huge drawback (partial workaround: use executemany() whenever possible). Psycopg 2 (as opposed to psycopg 1.1) does not support commit() on cursor (no serialized connections) Among the advantages of psycopg 2 is its very good support of mappings between Python data types and PostgreSQL data types. You can easily introduce your own mappings, even for composite data types. My $0.02 -- http://mail.python.org/mailman/listinfo/python-list
Re: Mod_python
Lad wrote: > In my web application I use Apache and mod_python. > I allow users to upload huge files( via HTTP FORM , using POST method) > I would like to store the file directly on a hard disk and not to > upload the WHOLE huge file into server's memory first. > Can anyone suggest a solution? The only solution you need is Apache and mod_python :) I mean, Apache won't load a huge POST request into its memory no matter what. All file uploads will be stored in temporary files. Under mod_python (provided you use FieldStorage) you'll need to deal only with 'file' objects. -- Maxim Sloyko -- http://mail.python.org/mailman/listinfo/python-list
lxml namespaces problem
Hi All! I have a little problem with XML namespaces. In my application I have two XML processors, that process the same document, one after the other. The first one looks for nodes in 'ns1' namespace, and substitutes them, according to some algorithm. After this processor is finished, it is guaranteed that there are no more 'ns1' nodes left in the tree. however 'ns1' namespace dclaration is still there, in the root node (well, I put it there manually). Now, when this namespace is no longer needed, I want to get rid of it, because it confuses some other processors (namely, my browser) So, the question is, how do I do that? del tree.getroot().nsmap['ns1'] does not seem to do the trick :( Thanks in advance and Happy Holidays! -- http://mail.python.org/mailman/listinfo/python-list
python 2.3.4, cx_Oracle 4.1 and utf-8 - trouble
Hello!
Trying to fetch long varchar2 column and get the following error:
cx_Oracle.DatabaseError: column at array pos 0 fetched with error: 1406
i.e. string buffer is not much enough to fetch the string.
# fragment of code...
myCon = cx_Oracle.connect(user, psw, dsn)
myCur = myCon.cursor()
myCur.execute("""
select COLUMN from TABLE where ID=1
""")
for record in myCur.fetchall():
# ...
Error is reproduced only if actual string value longer than half of
declared column size. For short strings all is ok.
If I print myCur.description, I get:
[('COLUMN', , 250, 250, 0, 0, 1)]
250 - declared column max size, but I guess cx_Oracle allocates only 250
bytes(!), so if my string longer than 125 chars (utf-8 national char
occupies > 1 byte) - I get the error.
Is it bug or what? Any suggestions?
Extra info:
OS - SuSE Linux 9.2
Client - Oracle Client 9.2.0.1.0
NLS_LANG="RUSSIAN_CIS.UTF8"
--
Best regards, Maxim Kuleshov
--
http://mail.python.org/mailman/listinfo/python-list
I can't inherit from "compiled" classes ?
Hello list, I'm trying to subclass socket and select, for both I get: """ TypeError: Error when calling the metaclass bases module.__init__() takes at most 2 arguments (3 given) """, I don't understand this error. Why would python try to pass 3 arguments (what are they) ? Googling for this error gave random results talking about try to inherit a "Package" but socket is definitely a class, (/usr/lib/python2.4/socket.py). Not sure about select thought. I've did the following to receive the error: """ In [1]: import socket In [2]: class PollingSocket(socket): ...: pass ...: --- exceptions.TypeError Traceback (most recent call last) /home/hq4ever/ TypeError: Error when calling the metaclass bases module.__init__() takes at most 2 arguments (3 given) """ What am I breaking wrong? Thank you, Maxim. -- Cheers, Maxim Veksler "Free as in Freedom" - Do u GNU ? -- http://mail.python.org/mailman/listinfo/python-list
Re: I can't inherit from "compiled" classes ?
On 4/29/07, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote: > In <[EMAIL PROTECTED]>, Maxim Veksler > wrote: > > > Hello list, > > > > I'm trying to subclass socket and select, for both I get: > > """ TypeError: Error when calling the metaclass bases > > module.__init__() takes at most 2 arguments (3 given) """, I don't > > understand this error. Why would python try to pass 3 arguments (what > > are they) ? > > > > Googling for this error gave random results talking about try to > > inherit a "Package" but socket is definitely a class, > > (/usr/lib/python2.4/socket.py). Not sure about select thought. > > > > I've did the following to receive the error: > > """ > > In [1]: import socket > > > > In [2]: class PollingSocket(socket): > >...: pass > >...: > > --- > > exceptions.TypeError Traceback (most > > recent call last) > > > > /home/hq4ever/ > > > > TypeError: Error when calling the metaclass bases > > module.__init__() takes at most 2 arguments (3 given) > > """ > > > > > > What am I breaking wrong? > > You are trying to subclass a module here, just like the error message > says. The module contains a `socket` type: > > In [3]: import socket > > In [4]: type(socket) > Out[4]: > > In [5]: type(socket.socket) > Out[5]: > Great, """ from socket import socket import select class PollingSocket(socket): pass """ > `select.select()` is a function: > > In [6]: import select > > In [7]: type(select.select) > Out[7]: > I understand what you are saying, and at the same time don't understand why it doesn't work. Isn't "everything an object" in python? And if something is an object does it not implies it's an instance of some class? Does this mean I can't somehow make this work: """class PollingSocket(socket.socket, select):""" ? Thanks for the help, > Ciao, > Marc 'BlackJack' Rintsch Maxim. -- Cheers, Maxim Veksler "Free as in Freedom" - Do u GNU ? -- http://mail.python.org/mailman/listinfo/python-list
Non blocking sockets with select.poll() ?
Hi,
I'm trying to write a non blocking socket port listener based on
poll() because select is limited to 1024 fd.
Here is the code, it never gets to "I did not block" until I do a
telnet connection to port 1.
"""
#!/usr/bin/env python
import socket
import select
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setblocking(0)
s.bind(('0.0.0.0', 1))
s.listen(5)
__poll = select.poll()
__poll.register(s)
__poll.poll()
print "I did not block"
"""
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Non blocking sockets with select.poll() ?
On 5/4/07, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
> On Fri, 4 May 2007 13:04:41 +0300, Maxim Veksler <[EMAIL PROTECTED]> wrote:
> >Hi,
> >
> >I'm trying to write a non blocking socket port listener based on
> >poll() because select is limited to 1024 fd.
> >
> >Here is the code, it never gets to "I did not block" until I do a
> >telnet connection to port 1.
> >
>
> What were you expecting?
>
I'll try to explain.
I'm doing a little experiment: Capturing the whole tcp 1-65535 range
of the machine, allowing me to connect to the my service on the
machine on every port. I know that it's probably the most dumb thing
to do with TCP/IP communication please don't forget it's an
experiment.
My first attempt was made with select.select please see here
http://article.gmane.org/gmane.comp.python.general/516155/ and the
next attempt for slice-by 1024 which also didn't work, here:
http://article.gmane.org/gmane.comp.python.general/517036/
Now, after I realized I won't be able to implement this using
select.select I turned to select.poll(). My problem now it that I'm
able to register more then 1024 socket fd but now I can't replicate
the behaviour select gave me. When I used select for <1024 sockets, I
was able to telnet to each of the 1024 ports and select.select would
return the proper object to do accept() on it, shortly speaking: no
exception was thrown, you can see the working code in the second link
above. The situation I have now with the code attached below is that
if I try querying one of the registered ports I get the following :
TERM1:
"""
./listener_sockets_range_poll.py
.
.
Asking 10182
Asking 10183
Asking 10184
Asking 10185
Asking 10186
Found 10186
Traceback (most recent call last):
File "./listener_sockets_range_poll.py", line 35, in
conn, addr = nb_active_socket.accept()
File "/usr/lib/python2.5/socket.py", line 167, in accept
sock, addr = self._sock.accept()
socket.error: (11, 'Resource temporarily unavailable')
"""
TERM2:
"""
telnet localhost 10100
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.
"""
and the code running is this:
"""
#!/usr/bin/env python
import socket
import select
class PollingSocket(socket.socket):
__poll = select.poll()
def __init__(self, port_number):
self.tcp_port_number = port_number
socket.socket.__init__(self, socket.AF_INET, socket.SOCK_STREAM)
self.setblocking(0)
self.bind(('0.0.0.0', self.tcp_port_number))
self.listen(5)
self.__poll.register(self)
def poll(self, timeout = 0):
return self.__poll.poll(timeout)
def debugPollingSocket(port_num):
print "BIND TO PORT: ", port_num
return PollingSocket(port_num)
all_sockets = map(debugPollingSocket, xrange(1, 19169))
print "We have this in stock:"
for nb_active_socket in all_sockets:
print nb_active_socket.tcp_port_number
while 1:
for nb_active_socket in all_sockets:
print "Asking", nb_active_socket.tcp_port_number
if nb_active_socket.poll(1):
print "Found", nb_active_socket.tcp_port_number
conn, addr = nb_active_socket.accept()
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()
"""
> Jean-Paul
Thank you,
Maxim.
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Non blocking sockets with select.poll() ?
On 5/4/07, Maxim Veksler <[EMAIL PROTECTED]> wrote:
> On 5/4/07, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
> > On Fri, 4 May 2007 13:04:41 +0300, Maxim Veksler <[EMAIL PROTECTED]> wrote:
> > >Hi,
> > >
> > >I'm trying to write a non blocking socket port listener based on
> > >poll() because select is limited to 1024 fd.
> > >
> > >Here is the code, it never gets to "I did not block" until I do a
> > >telnet connection to port 1.
> > >
> >
> > What were you expecting?
> >
>
> I'll try to explain.
> I'm doing a little experiment: Capturing the whole tcp 1-65535 range
> of the machine, allowing me to connect to the my service on the
> machine on every port. I know that it's probably the most dumb thing
> to do with TCP/IP communication please don't forget it's an
> experiment.
[snip]
I think I got it working now :)
"""
#!/usr/bin/env python
import socket
import select
class PollingSocket(socket.socket):
def __init__(self, port_number):
self.__poll = select.poll()
self.tcp_port_number = port_number
socket.socket.__init__(self, socket.AF_INET, socket.SOCK_STREAM)
self.setblocking(0)
self.bind(('0.0.0.0', self.tcp_port_number))
self.listen(5)
self.__poll.register(self)
def poll(self, timeout = 0):
return self.__poll.poll(timeout)
def debugPollingSocket(port_num):
print "BIND TO PORT: ", port_num
return PollingSocket(port_num)
all_sockets = map(debugPollingSocket, xrange(1, 19169))
print "We have this in stock:"
for nb_active_socket in all_sockets:
print nb_active_socket.tcp_port_number
while 1:
for nb_active_socket in all_sockets:
print "Asking", nb_active_socket.tcp_port_number
if nb_active_socket.poll(0):
print "Found", nb_active_socket.tcp_port_number
conn, addr = nb_active_socket.accept()
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()
"""
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Non blocking sockets with select.poll() ?
On 5/4/07, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
> >"""
> >#!/usr/bin/env python
> >import socket
> >import select
> >
> >class PollingSocket(socket.socket):
> >
> >
> >def __init__(self, port_number):
> >self.__poll = select.poll()
> >self.tcp_port_number = port_number
> >
> >socket.socket.__init__(self, socket.AF_INET, socket.SOCK_STREAM)
> >self.setblocking(0)
> >self.bind(('0.0.0.0', self.tcp_port_number))
> >self.listen(5)
> >self.__poll.register(self)
> >
> >def poll(self, timeout = 0):
> >return self.__poll.poll(timeout)
> >
> >def debugPollingSocket(port_num):
> >print "BIND TO PORT: ", port_num
> >return PollingSocket(port_num)
> >
> >all_sockets = map(debugPollingSocket, xrange(1, 19169))
> >
> >print "We have this in stock:"
> >for nb_active_socket in all_sockets:
> >print nb_active_socket.tcp_port_number
> >
> >while 1:
> >for nb_active_socket in all_sockets:
> >print "Asking", nb_active_socket.tcp_port_number
> >if nb_active_socket.poll(0):
> >print "Found", nb_active_socket.tcp_port_number
> >conn, addr = nb_active_socket.accept()
> >while 1:
> >data = conn.recv(1024)
> >if not data: break
> >conn.send(data)
> >conn.close()
> >"""
> >
>
> This will only handle one connection at a time, of course. The polling
> it does is also somewhat inefficient. Perhaps that's fine for your use
> case. If not, though, I'd suggest this version (untested):
>
Actually, I'm here to learn. I could have used any number of different
approaches to accomplish this; starting from
http://docs.python.org/lib/module-asyncore.html to twisted to
http://oss.coresecurity.com/projects/pcapy.html.
I would appreciate it if you could elaborate on why my loop is
inefficient, I will try to improve it then (and post back).
Besides, this whole story started from me writing a "quick
totalitarian" security testing framework. Once I'm done with the
networking part I will start working on the part that kill's all
current processes listening on TCP/IP of the machine. Obviously thats
not meant for production boxes... The simple idea is having the poller
on one side of the firewall connection and the "monster" on the other
side replying, a kind of primitive and plain firewall testing utility.
> from twisted.internet import pollreactor
> pollreactor.install()
>
> from twisted.internet import reactor
> from twisted.protocols.wire import Echo
> from twisted.internet.protocol import ServerFactory
>
> f = ServerFactory()
> f.protocol = Echo
> for i in range(1, 19169):
> reactor.listenTCP(i, f)
> reactor.run()
>
> This will handle traffic from an arbitrary number of clients at the same
> time and do so more efficiently than the loop in your version. You can
> also try epollreactor instead of pollreactor, if the version of Linux you
> are using supports epoll, for even better performance.
>
Thanks!
> Jean-Paul
>
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Init style output with python?
Hi list,
I'm working on writing sanity check script, and for aesthetic reasons
I would like the output be in the formatted like the gentoo init
script output, that is:
"""
Check for something .. [OK]
Check for something else ..[FAIL]
"""
Is there are frame work or something in python that would allow me to
do this (quickly) ?
If not, ideas how I should I be getting this boring task of:
1. get screen width
2. get output string length
3. get out status length
4. calculate space
5. print string, print space, print status, print newline
what happens if user changes textual terminal "resolution" ?
p.s.
I would also like to "OK" and "FAIL" output to be colored. I haven't
found anything for python to would allow to to output to ansi (linux,
rxvt, xterm). Here's a quick class I've written (in the hope it proves
to be useful to the next guy).
"""
#!/usr/bin/env python
""" This stuff is under GPL, as always"""
class ColorTerm:
def __init__(self, Mono = False):
pass
def __get_tput_color_value__(colorcode):
from commands import getoutput
return getoutput('tput setaf ' + colorcode)
BLACK_FG = __get_tput_color_value__('0')
RED_FG =__get_tput_color_value__('1')
GREEN_FG = __get_tput_color_value__('2')
YELLOW_FG = __get_tput_color_value__('3')
BLUE_FG = __get_tput_color_value__('4')
MAGENTA_FG =__get_tput_color_value__('5')
CYAN_FG = __get_tput_color_value__('6')
WHITE_FG = __get_tput_color_value__('7')
def black(self, msg):
return self.BLACK_FG + msg + self.BLACK_FG
def red(self, msg):
return self.RED_FG + msg + self.BLACK_FG
def green(self, msg):
return self.GREEN_FG + msg + self.BLACK_FG
def yellow(self, msg):
return self.YELLOW_FG + msg + self.BLACK_FG
def blue(self, msg):
return self.BLUE_FG + msg + self.BLACK_FG
def magenta(self, msg):
return self.MAGENTA_FG + msg + self.BLACK_FG
def cyan(self, msg):
return self.CYAN_FG + msg + self.BLACK_FG
def white(self, msg):
return self.WHITE_FG + msg + self.BLACK_FG
cc = ColorTerm()
print cc.red('Cool!') + cc.yellow('?'), cc.green('Sure is!!!')
print "Now setting your terminal text color to blue" + cc.BLUE_FG
print "well don't be blue about this, here let me set it back for you"
print cc.BLACK_FG + "see, nothing to worry about"
"""
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Init style output with python?
On 5/6/07, Tina I <[EMAIL PROTECTED]> wrote:
> Maxim Veksler wrote:
>
> > Is there are frame work or something in python that would allow me to
> > do this (quickly) ?
> > If not, ideas how I should I be getting this boring task of:
> > 1. get screen width
>
> You can look into the 'curses' module and do something like:
>
> screen = curses.initscreen()
> maxheight, maxwith = screen.getmaxyx()
>
> In my experience curses can be a bit tricky to work with but the online
> tutorials have some nice examples that help you avoid some of the
> pitfalls (like messing up your terminal)
>
> Tina
> --
> http://mail.python.org/mailman/listinfo/python-list
>
Fine! Thank you.
curses is very helpful, I'm attaching the code.
I see it has support for colors as well, but I haven't found any
tutorial that would explain how to use them.
Please note that this is just a draft, I'm not catching any
KeyboardInterrupt nor nothing.
"""#!/usr/bin/env python
class ColorTerm:
def __init__(self, Mono = False):
pass
def __get_tput_color_value__(colorcode):
from commands import getoutput
return getoutput('tput setaf ' + colorcode)
BLACK_FG = __get_tput_color_value__('0')
RED_FG =__get_tput_color_value__('1')
GREEN_FG = __get_tput_color_value__('2')
YELLOW_FG = __get_tput_color_value__('3')
BLUE_FG = __get_tput_color_value__('4')
MAGENTA_FG =__get_tput_color_value__('5')
CYAN_FG = __get_tput_color_value__('6')
WHITE_FG = __get_tput_color_value__('7')
def black(self, msg):
return self.BLACK_FG + msg + self.BLACK_FG
def red(self, msg):
return self.RED_FG + msg + self.BLACK_FG
def green(self, msg):
return self.GREEN_FG + msg + self.BLACK_FG
def yellow(self, msg):
return self.YELLOW_FG + msg + self.BLACK_FG
def blue(self, msg):
return self.BLUE_FG + msg + self.BLACK_FG
def magenta(self, msg):
return self.MAGENTA_FG + msg + self.BLACK_FG
def cyan(self, msg):
return self.CYAN_FG + msg + self.BLACK_FG
def white(self, msg):
return self.WHITE_FG + msg + self.BLACK_FG
class StatusWriter(ColorTerm):
import curses
def __init__(self, report_type = None):
pass
def initstyle_message(self, msg, status = True):
screen = self.curses.initscr(); self.curses.endwin()
if status:
status_msg = '[' + self.green('OK') + ']'
else:
status_msg = '[' + self.red('FAIL') + ']'
spaces_count = ( screen.getmaxyx()[1] - (len(msg)+len(status_msg)) )
return msg + ' '*spaces_count + status_msg
cc = StatusWriter()
while 1:
print cc.initstyle_message('The end is at hand')
print cc.initstyle_message('Lets party', False)
print cc.initstyle_message('Why like this?', True)
"""
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
programmatically manipulation environment variables of the calling shell
Hello list, I'm trying to write a python script that would allow me to manipulate shell variables of the calling shell. I'm trying to write some logic that would know to add LD_LIBRARY_PATH to the users environment. In bash this is done with "export", can I do this with python? (If at all possible because python is actually a sub process of bash). Thank you, Maxim. -- Cheers, Maxim Veksler "Free as in Freedom" - Do u GNU ? -- http://mail.python.org/mailman/listinfo/python-list
Re: programmatically manipulation environment variables of the calling shell
On 3/12/07, Tommy Nordgren <[EMAIL PROTECTED]> wrote: > > On 12 mar 2007, at 11.33, Maxim Veksler wrote: > > > Hello list, > > > > I'm trying to write a python script that would allow me to manipulate > > shell variables of the calling shell. I'm trying to write some logic > > that would know to add LD_LIBRARY_PATH to the users environment. > > > > In bash this is done with "export", can I do this with python? (If at > > all possible because python is actually a sub process of bash). > > > > Thank you, > > Maxim. > This is not possible to do directly. > There is however a possibility to solve this problem under special > circumstances. > 1. Your python (/perl/shell/etc) script must write the wanted > environment > variables to standard out, as key-value pairs with a suitable separator. > 2. The invoking shell must agree with your script as to what > separator is used. > 3. In the calling shell do myshellvars=`your_python_script`; > 4. Now the calling shell can parse the variable myshellvars to set > up the environment. Yes, thats exactly what I was set to do. <<< bash for export_val in $(python env_variables_maxim.py); do export $export_val; done >>> <<< python (env_variables_maxim.py) print 'FOO=123 BAR=/home/src:$BAR PATH=/nfs/bin' >>> Thank you. > -- > What is a woman that you forsake her, and the hearth fire and the > home acre, > to go with the old grey Widow Maker. --Kipling, harp song of the > Dane women > Tommy Nordgren > [EMAIL PROTECTED] > > > > -- Cheers, Maxim Veksler "Free as in Freedom" - Do u GNU ? -- http://mail.python.org/mailman/listinfo/python-list
low level networking in python
Hello, I wish to do some low level network stuff using python. I've googled somewhat and came up with pylibpcap[1], trouble is I can't compile it on my Ubuntu 6.10 workstation. Can someone please suggest a way to read some bits from random ports? I'm looking to write a simple pen-testing tool that would try from one side connecting to ports and from the other side sniff traffic to see on what ports traffic is coming. For pylibpcap I'm getting: """ [EMAIL PROTECTED]:~/development/personal/pylibpcap/pylibpcap-0.5.1$ python setup.py build running build running build_ext building '_pcapmodule' extension swig -python -shadow -ISWIG -o pcap.c pcap.i pcap.i:72: Warning(124): Specifying the language name in %typemap is deprecated - use #ifdef SWIG instead. pcap.i:77: Warning(124): Specifying the language name in %typemap is deprecated - use #ifdef SWIG instead. pcap.i:82: Warning(124): Specifying the language name in %typemap is deprecated - use #ifdef SWIG instead. /usr/bin/python ./build-tools/docify.py pcap.c /usr/bin/python ./build-tools/docify-shadow.py pcap.py Traceback (most recent call last): File "./build-tools/docify-shadow.py", line 30, in ? raise 'source file doesn\'t look like swigged shadow class code' source file doesn't look like swigged shadow class code error: command '/usr/bin/python' failed with exit status 1 """ [1] http://pylibpcap.sourceforge.net/ [2] http://py.vaults.ca/apyllo.py/126307487 -- Cheers, Maxim Veksler "Free as in Freedom" - Do u GNU ? -- http://mail.python.org/mailman/listinfo/python-list
Re: low level networking in python
On 3 Apr 2007 08:43:57 -0700, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> On Apr 3, 10:29 am, "Maxim Veksler" <[EMAIL PROTECTED]> wrote:
> > Hello,
> >
> > I wish to do some low level network stuff using python.
> >
> > --
> > Cheers,
> > Maxim Veksler
> >
> > "Free as in Freedom" - Do u GNU ?
>
> I would assume you could use the socket module. This post details
> someone else who opened ports with Python:
>
> http://www.thescripts.com/forum/thread44280.html
>
> Here's another resource using some python servers:
>
> http://docs.python.org/lib/socket-example.html
>
> Finally, a Socket programming howto:
> http://www.amk.ca/python/howto/sockets/
>
> I'm also told that the Twisted framework is excellent for this sort of
> thing.
>
Thanks for the heads-up. The sockets howto was great help.
I'm trying to bind a non-blocking socket, here is my code:
"""
#!/usr/bin/env python
import socket, select
from time import sleep
s_nb1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s_nb1.setblocking(0)
s_nb1.bind(('192.168.2.106', 10002))
s_nb1.listen(5)
while 1:
conn, addr = s_nb1.accept()
ready_to_read, ready_to_write, in_error = select.select([conn], [], [], 0)
print (ready_to_read, ready_to_write, in_error)
sleep(100)
s_nb1.close()
"""
And this is the exception I'm getting:
"""
python non_blocking_socket.py
Traceback (most recent call last):
File "non_blocking_socket.py", line 13, in ?
conn, addr = s_nb1.accept()
File "/usr/lib/python2.4/socket.py", line 161, in accept
sock, addr = self._sock.accept()
socket.error: (11, 'Resource temporarily unavailable')
"""
What am I doing wrong here?
p.s.
I've looked at twisted before posting this post. I've seen they
impelement alot of application level protocols but I didn't see much
treatment for low level "raw" network data, not to mention that it's a
way way over kill for what I'm asking to achieve. Twisted does have a
subproject called "Twisted Pair: Low-level networking" but sadly it's
unmaintained and undocumented.
> Mike
>
Maxim.
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: low level networking in python
On 4/4/07, Irmen de Jong <[EMAIL PROTECTED]> wrote:
> Maxim Veksler wrote:
>
> > I'm trying to bind a non-blocking socket, here is my code:
> > """
> > #!/usr/bin/env python
> >
> > import socket, select
> > from time import sleep
> >
> > s_nb1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> > s_nb1.setblocking(0)
> >
> > s_nb1.bind(('192.168.2.106', 10002))
> > s_nb1.listen(5)
> >
> > while 1:
> > conn, addr = s_nb1.accept()
> > ready_to_read, ready_to_write, in_error = select.select([conn], [],
> > [], 0)
> > print (ready_to_read, ready_to_write, in_error)
> > sleep(100)
> >
> > s_nb1.close()
> > """
> >
> > And this is the exception I'm getting:
> > """
> > python non_blocking_socket.py
> > Traceback (most recent call last):
> > File "non_blocking_socket.py", line 13, in ?
> >conn, addr = s_nb1.accept()
> > File "/usr/lib/python2.4/socket.py", line 161, in accept
> >sock, addr = self._sock.accept()
> > socket.error: (11, 'Resource temporarily unavailable')
> > """
> >
> > What am I doing wrong here?
>
> Nothing.
> Any operation on a non-blocking socket that is usually blocking
> (this includes accept(), bind(), connect(), recv with MSG_WAITALL)
> can possibly return a socket.error with errno set to EAGAIN.
> ('resource temporarily unavailable').
> If this happens you should use a select() on the socket to
> wait until it's done with the requested operation.
>
Hello everyone, I would like to thank you all for the helping tips so
far, with your help I managed to improve the previous code to not give
the error, I believe it's now working.
The non blocking echo socket code:
"""
#!/usr/bin/env python
import socket, select
s_nb1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s_nb1.setblocking(0)
s_nb1.bind(('0.0.0.0', 10002))
s_nb1.listen(5)
while 1:
ready_to_read, ready_to_write, in_error =
select.select([s_nb1], [], [], 0)
if s_nb1 in ready_to_read:
conn, addr = s_nb1.accept()
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()
s_nb1.close()
"""
> --Irmen
>
Maxim.
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
ulimit on open sockets ?
Hi,
I've written this code, the general idea was to listen on all 65535
port of tcp for connection.
"""
#!/usr/bin/env python
import socket, select
def get_non_blocking_socket(port_number):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setblocking(0)
s.bind(('0.0.0.0', port_number))
s.listen(1)
return s
all_sockets = map(get_non_blocking_socket, xrange(1, 15000))
while 1:
ready_to_read, ready_to_write, in_error =
select.select(all_sockets, [], [], 0)
for nb_active_socket in all_sockets:
if nb_active_socket in ready_to_read:
conn, addr = nb_active_socket.accept()
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()
"""
The thing is that when I tried to run this at first I got
"""
python non_blocking_range.py
Traceback (most recent call last):
File "non_blocking_range.py", line 12, in ?
all_sockets = map(get_non_blocking_socket, xrange(1, 15000))
File "non_blocking_range.py", line 6, in get_non_blocking_socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
File "/usr/lib/python2.4/socket.py", line 148, in __init__
_sock = _realsocket(family, type, proto)
socket.error: (24, 'Too many open files')
"""
So I set ulimit -n 50, now I'm getting
"""
python non_blocking_range.py
Traceback (most recent call last):
File "non_blocking_range.py", line 15, in ?
ready_to_read, ready_to_write, in_error =
select.select(all_sockets, [], [], 0)
ValueError: filedescriptor out of range in select()
"""
Should I be using a different version of select or something? Or
should I implement this the other way around, if so please suggest
how.
Thank you very much,
(enthusiastically learning python) Maxim.
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: ulimit on open sockets ?
On 4/10/07, Bjoern Schliessmann <[EMAIL PROTECTED]> wrote: > Maxim Veksler wrote: > > > I've written this code, the general idea was to listen on all > > 65535 port of tcp for connection. > > Please excuse the question: Why would anyone want to do such a manic > thing (instead of, e. g., using raw sockets)? > That's OK :) Well first because I can. Now to be a bit more serious, because I do stress testing on the system, because that's a handy utility to have (a "port catcher" to see how you application handles with that), because at the time I didn't knew about RAW sockets, because I want to do TCP connections from one side of the firewall to the other and implementing the TCP stacks by my self (If I choose to go the raw socket way) is something I wouldn't like to do (at least for now). More... > Regards, > > > Björn > Maxim. > -- > BOFH excuse #326: > > We need a licensed electrician to replace the light bulbs in the > computer room. > > -- > http://mail.python.org/mailman/listinfo/python-list > -- Cheers, Maxim Veksler "Free as in Freedom" - Do u GNU ? -- http://mail.python.org/mailman/listinfo/python-list
Re: ulimit on open sockets ?
On 4/10/07, Alex Martelli <[EMAIL PROTECTED]> wrote:
> Maxim Veksler <[EMAIL PROTECTED]> wrote:
>
> > ValueError: filedescriptor out of range in select()
> > """
> >
> > Should I be using a different version of select or something? Or
>
> select typically supports 1024 FDs at most (a design limit of the
> underlying operating system). You may want to try poll instead (epoll
> might be better but I doubt Python supports it yet).
>
I've read some post the other day of a guy faced similar problem and
it turns out {e,}poll is limited as well, besides I don't know how to
use it so an example would be great.
Now, someone I work with suggested a simple work around "Pass the list
objects in groups of 1024 each time to the select.select structure". I
think it's acceptable and good advice, the thing is I don't know how
to implement this "the python way" (that is - with out it being ugly).
Can I do modulation ( % 1024 ) on the main iterator loop?
Something like:
for nb_active_socket in (all_sockets % 1024):
if nb_active_socket in ready_to_read:
conn, addr = nb_active_socket.accept()
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()
?
Thanks for helping,
Maxim.
>
> Alex
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: ulimit on open sockets ?
On 4/13/07, Alex Martelli <[EMAIL PROTECTED]> wrote:
>
> On Apr 12, 2007, at 1:17 PM, Maxim Veksler wrote:
> ...
> > Now, someone I work with suggested a simple work around "Pass the list
> > objects in groups of 1024 each time to the select.select structure". I
> > think it's acceptable and good advice, the thing is I don't know how
> > to implement this "the python way" (that is - with out it being ugly).
>
> I don't understand how you're going to make it work (I see no select
> calls in your code and I don't understand how you'd get one in there
> by polling), but I'm going to just explain how to get slices of 1024
> items at a time from a long list.
>
Thank you. I'm attaching the full code so far for reference, sadly it
still doesn't work. It seems that select.select gets it's count of
fd's not from the amount passed to it by the sub_list but from the
kernel (or whatever) count for the process; The main issue here is
that it seems I won't be able to use select for the simple
non-blocking process and am forced to check poll to see if that helps.
The error I'm getting is still the same:
# ulimit -n
50
# python listener_sockets_range.py
Traceback (most recent call last):
File "listener_sockets_range.py", line 22, in ?
ready_to_read, ready_to_write, in_error =
select.select(select_cap_sockets, [], [], 0)
ValueError: filedescriptor out of range in select()
"""
#!/usr/bin/env python
import socket, select
def get_non_blocking_socket(port_number):
print port_number
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setblocking(0)
s.bind(('0.0.0.0', port_number))
s.listen(1)
return s
def slice_by_fd_limit(longlist, N=1024):
for i in xrange(0, len(longlist), N):
yield longlist[i:i+N]
all_sockets = map(get_non_blocking_socket, xrange(1, 2))
while 1:
for select_cap_sockets in slice_by_fd_limit(all_sockets):
ready_to_read, ready_to_write, in_error =
select.select(select_cap_sockets, [], [], 0)
for nb_active_socket in all_sockets:
if nb_active_socket in ready_to_read:
conn, addr = nb_active_socket.accept()
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()
"""
--
Cheers,
Maxim Veksler
"Free as in Freedom" - Do u GNU ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Thoughts on using isinstance
On Jan 24, 3:38 pm, "abcd" <[EMAIL PROTECTED]> wrote: > In my code I am debating whether or not to validate the types of data > being passed to my functions. For example > > def sayHello(self, name): > if not name: > rasie "name can't be null" > if not isinstance(name, str): > raise "name must be a string" > print "Hello " + name > > Is the use of isinstance a "bad" way of doing things? is it a "heavy" > operation? for example, if I use this in each function validate input > will it slow things down a lot? > > just curious how you might handle this type of situation (other than > not validating at all). > > thanks My opinion is that validation is generally good. However, you have to make it not too strict. For example, instead of print "Hello " + name you could have written print "Hello " + str(name) In this case requirement isinstance() will be too strict. The only thing you have to check is that hasattr(name, "__str__") and callable(name.__str__) In this case you can have validation, while at the same time enjoy full flexibility of dynamic typing. -- Maxim -- http://mail.python.org/mailman/listinfo/python-list
lambda functions ?
Hello, I'm new on this list and in python. It seems python has some interesting concept of "ad hoc" function which I'm trying to understand without much success. Take the following code for example: """ >>> def make_incrementor(n): ... return lambda x: x + n ... >>> f = make_incrementor(42) >>> f(0) 42 >>> f(1) 43 """ I really don't understand whats going on here. On the first instantiating of the object "f" where does "x" gets it's value? Or is it evaluated as 0? ie "x: 0 + 42" And what is the "f" object? An integer? a pointer? an Object? I'm coming from the C world... Could some please try (if even possible) to implement the above code without using "lambda" I believe it would help me grasp this a bit faster then. Thank you, Maxim. -- Cheers, Maxim Veksler "Free as in Freedom" - Do u GNU ? -- http://mail.python.org/mailman/listinfo/python-list
Re: lambda functions ?
Wow, Thank you everyone for the help. I am amazed by the motivation people have on this list to help new comers. I hope that I will be able to contribute equally some day. On 05 Feb 2007 14:22:05 -0800, Paul Rubin <"http://phr.cx"@nospam.invalid> wrote: > "Maxim Veksler" <[EMAIL PROTECTED]> writes: > > >>> def make_incrementor(n): > > ... return lambda x: x + n > > Is the same as: > > def make_incrementor(n): > def inner(x): > return x + n > return inner > > When you enter make_incrementor, it allocates a memory slot (normally > we'd think of this as a stack slot since it's a function argument, but > it's really in the heap) and copies its argument there. Then you > create the function "inner" which refers to that memory slot because > of the reference to "n". Then make_incrementor returns, but since the > returned object still contains the reference to that memory slot, the > memory slot is not freed (this is the part where it becomes important > that the slot is really not on the stack). So when you call the > returned function, it still can get at that slot. > Following the debugger on your code, I can identify the following stages: def make_incrementor(n): def inner(x): return x + n return inner f = make_incrementor(10) f(10) f(0) 1. "def make_incrementor(n)" Create function object in memory and set "make_incrementor" to point to it. 2. "f = make_incrementor(10)" Set "f" to point to "inner(x) function". Set n value to 10. 3. "f(10)" Call to inner(x), which will return "father" n + "local" x. This means that "f" is not a pointer to make_incrementor but rather to the internal (copied?) function. > This style is very common in Scheme programming so you might read a > Scheme book if you want to understand it. The classic: > > http://mitpress.mit.edu/sicp/ > I might just well do that. > > Could some please try (if even possible) to implement the above code > > without using "lambda" I believe it would help me grasp this a bit > > faster then. > > Does the above help? all Your explanation was excellent. Thank you. > -- > http://mail.python.org/mailman/listinfo/python-list > -- Cheers, Maxim Veksler "Free as in Freedom" - Do u GNU ? -- http://mail.python.org/mailman/listinfo/python-list
Binary search tree
Hi, I have to get list of URLs one by one and to find the URLs that I have more than one time(can't be more than twice). I thought to put them into binary search tree, this way they'll be sorted and I'll be able to check if the URL already exist. Couldn't find any python library that implements trees. Is there some library of this kind in python? Or can I find it somewhere else? -- http://mail.python.org/mailman/listinfo/python-list
Re: ZSI usage
There is a typo there in functions name. It is called "session_open" not "open_session", but everything else is as described -- http://mail.python.org/mailman/listinfo/python-list
ZSI usage
Hi all!
I'm trying to make a simple SOAP call from python to SOAP::Lite (perl)
SOAP server.
My SOAP server has https://myserv.com/open-api URI, the function
open_session has the "QW/API" namespace. SO I do the following:
from ZSI.client import Binding
fp = open('debug.out', 'a')
client = Binding(url='/open-api',
host='myserv.com',
port=443,
ssl=1, tracefile=fp)
client.SetNS("QW/API")
sid = client.open_session(1)
However, I get the following error:
ZSI.FaultException: SOAPAction shall match 'uri#method' if present (got
'""', expected 'QW/API#open_session'
If I look at SOAP packet I see the following:
[snip]
1
[snip]
SO the question is, why ZSI seem to ignore that NS setting? How to fix
that?
Thanks in advance!
--
Maxim
--
http://mail.python.org/mailman/listinfo/python-list
Re: Best way to handle exceptions with try/finally
I guess the following standard method will help : class MyLocker(object): def __init__(self, lock): self.lock = lock self.lock.acquire() def __del__(self): self.lock.release() Then whenever you need to acquire a lock: templock = MyLocker(self.__mutex) del templock # will release the lock (provided you didn't create an extra link to this object) -- http://mail.python.org/mailman/listinfo/python-list
py2exe and distutils
Hi, i have installed Python 2.5.4 on WinXP, setuptools-0.6c9 and py2exe 0.6.9 Now i can't list installed modules, here is the stacktrace: help> modules Please wait a moment while I gather a list of all available modules... Traceback (most recent call last): File "", line 1, in File "C:\Programme\Python25\lib\site.py", line 346, in __call__ return pydoc.help(*args, **kwds) File "C:\Programme\Python25\lib\pydoc.py", line 1649, in __call__ self.interact() File "C:\Programme\Python25\lib\pydoc.py", line 1667, in interact self.help(request) File "C:\Programme\Python25\lib\pydoc.py", line 1683, in help elif request == 'modules': self.listmodules() File "C:\Programme\Python25\lib\pydoc.py", line 1804, in listmodules ModuleScanner().run(callback) File "C:\Programme\Python25\lib\pydoc.py", line 1855, in run for importer, modname, ispkg in pkgutil.walk_packages(): File "C:\Programme\Python25\lib\pkgutil.py", line 110, in walk_packages __import__(name) File "C:\Programme\Python25\Lib\site-packages\setuptools\__init__.py", line 2, in from setuptools.extension import Extension, Library File "C:\Programme\Python25\Lib\site-packages\setuptools\extension.py", line 2, in from dist import _get_unpatched File "C:\Programme\Python25\Lib\site-packages\setuptools\dist.py", line 27, in _Distribution = _get_unpatched(_Distribution) File "C:\Programme\Python25\Lib\site-packages\setuptools\dist.py", line 23, in _get_unpatched "distutils has already been patched by %r" % cls AssertionError: distutils has already been patched by py2exe.Distribution at 0x011B4F90> Any suggestion, how to fix this issue? Best regards Maxim -- http://mail.python.org/mailman/listinfo/python-list
Re: py2exe and distutils
Thomas Heller schrieb: Maxim Demenko schrieb: Hi, i have installed Python 2.5.4 on WinXP, setuptools-0.6c9 and py2exe 0.6.9 Now i can't list installed modules, here is the stacktrace: [...] Any suggestion, how to fix this issue? Thomas Heller schrieb: Looks like a setuptools problem to me. Here's the output on my system: Actually, I don't know where the problem is. Maybe pydoc? Thomas Thank you Thomas, i found http://thread.gmane.org/gmane.comp.python.distutils.devel/3340 and tried to import setuptools first - indeed, in this case the problem seems to be solved, however, would like to know, how to persist it. If i put it into py2exe.__init__.py - is it a very bad idea? Best regards Maxim -- http://mail.python.org/mailman/listinfo/python-list
Re: py2exe and distutils
Gabriel Genellina schrieb: En Sat, 07 Feb 2009 18:39:19 -0200, Thomas Heller escribió: Maxim Demenko schrieb: Hi, i have installed Python 2.5.4 on WinXP, setuptools-0.6c9 and py2exe 0.6.9 Now i can't list installed modules, here is the stacktrace: [...] Any suggestion, how to fix this issue? Thomas Heller schrieb: Looks like a setuptools problem to me. Here's the output on my system: Actually, I don't know where the problem is. Maybe pydoc? Yes, pydoc isn't robust enough in 2.5; see <http://groups.google.com/group/comp.lang.python/browse_thread/thread/4d07c1f7bdb49b94/1bdd47421061c788#1bdd47421061c788> for a quick fix. Thank you Gabriel, that works nicely. Best regards Maxim -- http://mail.python.org/mailman/listinfo/python-list
Strange array.array performance
Hello all,
I'm currently writing a Python <-> MATLAB interface with ctypes and
array.array class, using which I'll need to push large amounts of data
to MATLAB. Everything is working well, but there was one strange
performance-related issue that I ran into and wanted to ask about.
Here's some example code to illustrate my point (this is developed on
Windows, hence the use of clock):
---
from array import array
from time import clock
input = array('B', range(256) * 1)
# Case 1
start = clock()
data1 = array('B', input)
print format(clock() - start, '.10f')
# Case 2
start = clock()
data2 = array('B')
data2[:] = input
print format(clock() - start, '.10f')
# Case 3
start = clock()
data3 = array('B')
data3.extend(input)
print format(clock() - start, '.10f')
print input == data1 == data2 == data3
---
The output from this on my machine is as follows:
0.7080547730
0.0029827034
0.0028685943
True
That seems very wrong. In the end, all arrays have the same data, but
by specifying it in the constructor the creation process takes over
350x longer than the other two methods. Is this a bug, or is there
some valid reason for it?
In the latter case, it would be a good idea to mention this in the
documentation, since that can be a significant performance improvement
in some applications. Currently the documentation states "Otherwise,
the iterable initializer is passed to the extend() method," which
doesn't seem to be the case, based on the third example.
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange array.array performance
On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern wrote:
> On 2009-02-19 12:52, Maxim Khitrov wrote:
>>
>> Hello all,
>>
>> I'm currently writing a Python<-> MATLAB interface with ctypes and
>> array.array class, using which I'll need to push large amounts of data
>> to MATLAB.
>
> Have you taken a look at mlabwrap?
>
> http://mlabwrap.sourceforge.net/
>
> At the very least, you will probably want to use numpy arrays instead of
> array.array.
>
> http://numpy.scipy.org/
I have, but numpy is not currently available for python 2.6, which is
what I need for some other features, and I'm trying to keep the
dependencies down in any case. Mlabwrap description doesn't mention if
it is thread-safe, and that's another one of my requirements.
The only feature that I'm missing with array.array is the ability to
quickly pre-allocate large chunks of memory. To do that right now I'm
using array('d', (0,) * size). It would be nice if array accepted an
int as the second argument indicating how much memory to allocate and
initialize to 0.
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange array.array performance
On Thu, Feb 19, 2009 at 2:23 PM, Gabriel Genellina
wrote:
> En Thu, 19 Feb 2009 16:52:54 -0200, Maxim Khitrov
> escribió:
>
>> input = array('B', range(256) * 1)
>>
>> # Case 1
>> start = clock()
>> data1 = array('B', input)
>> print format(clock() - start, '.10f')
>
>> That seems very wrong. In the end, all arrays have the same data, but
>> by specifying it in the constructor the creation process takes over
>> 350x longer than the other two methods. Is this a bug, or is there
>> some valid reason for it?
>
> It's a known issue: http://bugs.python.org/issue5109
I see, thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange array.array performance
On Thu, Feb 19, 2009 at 7:01 PM, Scott David Daniels
wrote:
> Maxim Khitrov wrote:
>>
>> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern
>> wrote:
>> I have, but numpy is not currently available for python 2.6, which is
>> what I need for some other features, and I'm trying to keep the
>> dependencies down in any case
>> The only feature that I'm missing with array.array is the ability to
>> quickly pre-allocate large chunks of memory. To do that right now I'm
>> using array('d', (0,) * size). It would be nice if array accepted an
>> int as the second argument indicating how much memory to allocate and
>> initialize to 0.
>
> In the meantime, you could write a function (to ease the shift to numpy)
> and reduce your interface problem to a very small set of lines:
>def zeroes_d(n):
>'''Allocate a n-element vector of 'd' elements'''
>vector = array.array('d') # fromstring has no performance bug
>vector.fromstring(n * 8 * '\0')
>return vector
> Once numpy is up and running on 2.6, this should be easy to convert
> to a call to zeroes.
If I do decide to transition at any point, it will require much
greater modification. For example, to speed-up retrieval of data from
Matlab, which is returned to me as an mxArray structure, I allocate an
array.array for it and then use ctypes.memmove to copy data directly
into the array's buffer (address obtained through buffer_info()).
Same thing for sending data, rather than allocate a separate mxArray,
copy data, and then send, I create an empty mxArray and set its data
pointer to the array's buffer. I'm sure that there are equivalents in
numpy, but the point is that the transition, which currently would not
benefit my code in any significant way, will not be a quick change.
On the other hand, I have to thank you for the fromstring example. For
some reason, it never occurred to me that creating a string of nulls
would be much faster than a tuple of zeros. In fact, you can pass the
string to the constructor and it calls fromstring automatically. For
an array of 1 million elements, using a string to initialize is 18x
faster. :)
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange array.array performance
On Thu, Feb 19, 2009 at 7:01 PM, Scott David Daniels
wrote:
> Maxim Khitrov wrote:
>>
>> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern
>> wrote:
>> I have, but numpy is not currently available for python 2.6, which is
>> what I need for some other features, and I'm trying to keep the
>> dependencies down in any case
>> The only feature that I'm missing with array.array is the ability to
>> quickly pre-allocate large chunks of memory. To do that right now I'm
>> using array('d', (0,) * size). It would be nice if array accepted an
>> int as the second argument indicating how much memory to allocate and
>> initialize to 0.
>
> In the meantime, you could write a function (to ease the shift to numpy)
> and reduce your interface problem to a very small set of lines:
>def zeroes_d(n):
>'''Allocate a n-element vector of 'd' elements'''
>vector = array.array('d') # fromstring has no performance bug
>vector.fromstring(n * 8 * '\0')
>return vector
> Once numpy is up and running on 2.6, this should be easy to convert
> to a call to zeroes.
Here's the function that I'll be using from now on. It gives me
exactly the behavior I need, with an int initializer being treated as
array size. Still not as efficient as it could be if supported
natively by array (one malloc instead of two + memmove + extra
function call), but very good performance nevertheless:
from array import array as _array
array_null = dict((tc, '\0' * _array(tc).itemsize) for tc in 'cbBuhHiIlLfd')
def array(typecode, init):
if isinstance(init, int):
return _array(typecode, array_null[typecode] * init)
return _array(typecode, init)
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange array.array performance
On Thu, Feb 19, 2009 at 9:34 PM, David Cournapeau wrote: > On Fri, Feb 20, 2009 at 4:53 AM, Maxim Khitrov wrote: >> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern wrote: >>> On 2009-02-19 12:52, Maxim Khitrov wrote: >>>> >>>> Hello all, >>>> >>>> I'm currently writing a Python<-> MATLAB interface with ctypes and >>>> array.array class, using which I'll need to push large amounts of data >>>> to MATLAB. >>> >>> Have you taken a look at mlabwrap? >>> >>> http://mlabwrap.sourceforge.net/ >>> >>> At the very least, you will probably want to use numpy arrays instead of >>> array.array. >>> >>> http://numpy.scipy.org/ >> >> I have, but numpy is not currently available for python 2.6, which is >> what I need for some other features, and I'm trying to keep the >> dependencies down in any case. Mlabwrap description doesn't mention if >> it is thread-safe, and that's another one of my requirements. > > What do you mean by thread-safe ? Different threads calling the same > matlab engine handle ? Yes, I may have a case where one thread is still sending data, while another tries to close the connection, or two threads trying to close the connection at the same time. In both cases, I need some parts of the code to be atomic to prevent errors. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange array.array performance
On Thu, Feb 19, 2009 at 9:15 PM, John Machin wrote:
> On Feb 20, 6:53 am, Maxim Khitrov wrote:
>> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern wrote:
>> > On 2009-02-19 12:52, Maxim Khitrov wrote:
>>
>> >> Hello all,
>>
>> >> I'm currently writing a Python<-> MATLAB interface with ctypes and
>> >> array.array class, using which I'll need to push large amounts of data
>> >> to MATLAB.
>>
>> > Have you taken a look at mlabwrap?
>>
>> > http://mlabwrap.sourceforge.net/
>>
>> > At the very least, you will probably want to use numpy arrays instead of
>> > array.array.
>>
>> > http://numpy.scipy.org/
>>
>> I have, but numpy is not currently available for python 2.6, which is
>> what I need for some other features, and I'm trying to keep the
>> dependencies down in any case. Mlabwrap description doesn't mention if
>> it is thread-safe, and that's another one of my requirements.
>>
>> The only feature that I'm missing with array.array is the ability to
>> quickly pre-allocate large chunks of memory. To do that right now I'm
>> using array('d', (0,) * size).
>
> It would go somewhat faster if you gave it a float instead of an int.
>
>> It would be nice if array accepted an
>> int as the second argument indicating how much memory to allocate and
>> initialize to 0.
>
> While you're waiting for that to happen, you'll have to use the
> fromstring trick, or another gimmick that is faster and is likely not
> to use an extra temp 8Mb for a 1M-element array, as I presume the
> fromstring does.
>
> [Python 2.6.1 on Windows XP SP3]
> [Processor: x86 Family 15 Model 36 Stepping 2 AuthenticAMD ~1994 Mhz]
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
> ('d',(0,)*
> 100)"
> 10 loops, best of 3: 199 msec per loop
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
> ('d',(0.,)*100)"
> 10 loops, best of 3: 158 msec per loop
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
> ('d');x.fromstring('\0'*8*100)"
> 10 loops, best of 3: 36 msec per loop
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
> ('d','\0'*8*100)"
> 10 loops, best of 3: 35.7 msec per loop
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "array
> ('d',(0.,))*100"
> 10 loops, best of 3: 19.5 msec per loop
Interesting, though I'm not able to replicate that last outcome. The
string method is still the fastest on my machine. Furthermore, it
looks like the order in which you do the multiplication also matters -
(8 * size * '\0') is faster than ('\0' * 8 * size). Here is my test
and outcome:
---
from array import array
from timeit import repeat
print repeat(lambda: array('d', (0,) * 10), number = 100)
print repeat(lambda: array('d', (0.0,) * 10), number = 100)
print repeat(lambda: array('d', (0.0,)) * 10, number = 100)
print repeat(lambda: array('d', '\0' * 10 * 8), number = 100)
print repeat(lambda: array('d', '\0' * 8 * 10), number = 100)
print repeat(lambda: array('d', 8 * 10 * '\0'), number = 100)
---
[0.91048107424534941, 0.88766983642377162, 0.88312824645684618]
[0.72164595848486179, 0.72038338197219343, 0.72346024633711981]
[0.10763947529894136, 0.1047547164728595, 0.10461521722863232]
[0.05856873793382178, 0.058508825334111947, 0.058361838698573365]
[0.057632016342657799, 0.057521392119007864, 0.057227118035289237]
[0.056006643320014149, 0.056331811311153501, 0.05618743321510]
The array('d', (0.0,)) * 10 method is a good compromise between
performance and amount of memory used, so maybe I'll use that instead.
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange array.array performance
On Thu, Feb 19, 2009 at 10:06 PM, David Cournapeau wrote: > On Fri, Feb 20, 2009 at 11:43 AM, Maxim Khitrov wrote: >> >> Yes, I may have a case where one thread is still sending data, while >> another tries to close the connection, or two threads trying to close >> the connection at the same time. In both cases, I need some parts of >> the code to be atomic to prevent errors. > > That does not sound like the right approach, then. Matlab engine is > not thread safe in that sense: > > http://www.mathworks.fr/support/solutions/data/1-YR98I.html?product=ML&solution=1-YR98I > > cheers, > > David > "One option is to use MUTEXes around every call into the MATLAB Engine" - that's exactly what I'm doing. Some setup work, like creating mxArrays is done without a lock, but all eng* calls are mutually exclusive. I've been doing a lot of my own testing, and so far have seen no problems with this approach. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange array.array performance
On Fri, Feb 20, 2009 at 2:42 AM, Scott David Daniels wrote: > Maxim Khitrov wrote: >> >> ... Here's the function that I'll be using from now on. It gives me >> exactly the behavior I need, with an int initializer being treated as >> array size. Still not as efficient as it could be if supported >> natively by array (one malloc instead of two + memmove + extra >> function call), but very good performance nevertheless: >> >> from array import array as _array >> array_null = dict((tc, '\0' * _array(tc).itemsize) for tc in >> 'cbBuhHiIlLfd') > > How about: > array_null = dict((tc, _array(tc, (0,)).tostring() for tc in > 'cbBuhHiIlLfd') > ... > (some ancient floating points did not use all-0 bytes for 0.0). Didn't know that, thanks. I actually got rid of the dict, since benchmarks showed access time to itemsize in the function itself is not any slower than dict access. After going through all the different speed tests yesterday the function now looks like this: from array import array as _array def array(typecode, init): if isinstance(init, int): a = _array(typecode, (0,)) if a.itemsize * init > 1048576: return a * init else: a.fromstring((init - 1) * a.tostring()) return a return _array(typecode, init) This uses the fast fromstring operation when creating an array that is less than 1MB in size. Over that, array multiplication is used, which is slower, but doesn't require the extra memory. - Max -- http://mail.python.org/mailman/listinfo/python-list
Using clock() in threading on Windows
Greetings, The threading module uses time.time in _Condition and _Thread classes to implement timeouts. On Windows, time() typically has a resolution of 15.625ms. In addition, if the system clock is changed (though ntp, for example) it would reflect that change, causing the timeout to last longer or shorter depending on which way the update went. Would it not be better to use time.clock() instead? The resolution is much better, and the value is not linked to system clock. Right now, I replace the threading._time reference with clock in some of my programs and everything works perfectly. Condition and Event timeouts are precise down to the millisecond (resolution of the sleep function), and I see no side-effects. Is it possible to make that change part of the module itself (keeping time() for linux systems), or can someone think of a reason why using clock is a bad idea? I know that it's using QueryPerformanceCounter for implementation, which has some known issues, but I still think that the advantages outweigh potential faults. - Max -- http://mail.python.org/mailman/listinfo/python-list
What functions, other than sleep(), can be interrupted by Ctrl-C?
Greetings, I'm looking for a function in the standard library or pywin32 package that will block until a certain condition is met or it is interrupted by Ctrl-C. For example, time.sleep() would have been perfect for my needs if thread.interrupt_main() could interrupt the call from another thread in the same way that Ctrl-C does. Unfortunately, that is not the case. Another thing I tried was creating a pipe with os.pipe() and issuing a read call on it. The event to exit was a single byte written to the other end of the pipe, but Ctrl-C could not interrupt the read call. The threading.Event class does not work for me, because it uses short sleep intervals for timed waits. I need the reaction to be as close to instant as possible, something like this will not do: while not : sleep(0.01) I actually replaced threading.Event with my own version that uses native Windows events, and waits on those also cannot be interrupted by Ctrl-C. I'm trying to achieve the same effect as that while loop and I don't care what the condition to exit is, but the loop needs to exit as soon as the condition is met without waiting for up to X additional milliseconds. Any ideas? - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: What functions, other than sleep(), can be interrupted by Ctrl-C?
On Thu, Feb 26, 2009 at 3:47 PM, Gabriel Genellina wrote: >> I'm looking for a function in the standard library or pywin32 package >> that will block until a certain condition is met or it is interrupted >> by Ctrl-C. For example, time.sleep() would have been perfect for my >> needs if thread.interrupt_main() could interrupt the call from another >> thread in the same way that Ctrl-C does. Unfortunately, that is not >> the case. > > You may try MsgWaitForMultipleObjects - send a message to the main thread > from the other thread. > An alertable wait (like SleepEx) plus QueueUserAPC should work, I presume, > but I've never actually tried in Python. I tried using MsgWaitForMultipleObjects, but even with a wake mask of 0x Ctrl-C still did not interrupt the call. Maybe it has to do with the way pywin32 implements it. QueueUserAPC is not available in pywin32, but I suppose I can try getting to it via ctypes. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: py2exe automatic upgrades of a program while it is running, is that possible?
On Mon, Mar 2, 2009 at 9:18 PM, William Heath wrote: > Hi All, > I am using py2exe to create a windows executable. I am curious if anyone > knows a way to automatically upgrade a py2exe windows executable while it is > running. Is that possible? If so how? If it isn't possible, what is the > next best thing? Also, if it is not available using py2exe is it available > in other languages/solutions your aware of? > -Tim I don't think there is a way to do this directly. The best thing that I can think of is to have your program to use the subprocess or multiprocessing modules to launch a second instance of itself when an update is required. The parent would pass the child any necessary data so the child could continue where the parent left off, have child acknowledge this transaction, and then the parent can exit. Depending on what your program is actually doing this may not be a reasonable solution. Look into os.pipe, multiprocessing.Queue, and subprocess.Popen.communicate. If you have pywin32 extension installed, win32event module contains a number of useful functions for things like creating inter-process events. - Max -- http://mail.python.org/mailman/listinfo/python-list
Style question - defining immutable class data members
Very simple question on the preferred coding style. I frequently write classes that have some data members initialized to immutable values. For example: class Test(object): def __init__(self): self.some_value = 0 self.another_value = None Similar effect can be achieved by defining some_value and another_value for the entire class, like so: class Test(object): some_value = 0 another_value = None The advantage of doing this is that the assignments are evaluated once and thus the creation of that class is a bit faster. Access is still performed through self.some_value and self.another_value. Is there a reason to prefer the first style over the second? - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Style question - defining immutable class data members
On Sat, Mar 14, 2009 at 12:50 PM, MRAB wrote: > Maxim Khitrov wrote: >> >> Very simple question on the preferred coding style. I frequently write >> classes that have some data members initialized to immutable values. >> For example: >> >> class Test(object): >> def __init__(self): >> self.some_value = 0 >> self.another_value = None >> >> Similar effect can be achieved by defining some_value and >> another_value for the entire class, like so: >> >> class Test(object): >> some_value = 0 >> another_value = None >> >> The advantage of doing this is that the assignments are evaluated once >> and thus the creation of that class is a bit faster. Access is still >> performed through self.some_value and self.another_value. Is there a >> reason to prefer the first style over the second? >> > In the first case each instance has its own attributes, whereas in the > second case the attributes belong to the class and are thus shared by > all the instances. Which you use depends on whether you want them shared > or not. When the types are immutable, there is no difference. The second case is faster and consumes less memory (initially), but nothing else changes. I'm simply asking if it is considered poor style to define data members for the class when they are only used by instances. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Style question - defining immutable class data members
On Sat, Mar 14, 2009 at 2:07 PM, Gary Herron wrote: > Maxim Khitrov wrote: >> >> Very simple question on the preferred coding style. I frequently write >> classes that have some data members initialized to immutable values. >> For example: >> >> class Test(object): >> def __init__(self): >> self.some_value = 0 >> self.another_value = None >> >> Similar effect can be achieved by defining some_value and >> another_value for the entire class, like so: >> >> class Test(object): >> some_value = 0 >> another_value = None >> >> The advantage of doing this is that the assignments are evaluated once >> and thus the creation of that class is a bit faster. Access is still >> performed through self.some_value and self.another_value. Is there a >> reason to prefer the first style over the second? >> >> - Max >> -- >> http://mail.python.org/mailman/listinfo/python-list >> > > Such things are often called class attributes, and the are fine. If you > look through Python's standard library, you will find many examples of > class attributes. > > However, you appear to have either a misuse or misconception of the word > "immutable' here. Whether the value you assign to a class attribute is > mutable or immutable is irrelevant. Also whether you plan on leaving the > value constant or not is also not relevant. > What does matter is this: If every instance wants access to a single value > (immutable or not), use a class attribute, otherwise use an instance > attribute. > > Gary Herron Perhaps a different example would help explain what I'm trying to do: class Case1(object): def __init__(self): self.count = 0 self.list = [] def inc(self): self.count += 1 self.list.append(self.count) def val(self): return (self.count, self.list) class Case2(object): count = 0 list = [] def inc(self): self.count += 1 self.list.append(self.count) def val(self): return (self.count, self.list) for i in xrange(10): c1 = Case1() c2 = Case2() for j in xrange(i): c1.inc() c2.inc() v1, l1 = c1.val() v2, l2 = c2.val() print v1 == v2, l1 == l2 The only difference between Case1 and Case2 classes is where the count and list attributes are defined. You will notice that for an immutable type (count), this doesn't matter. On the last line, v1 == v2 is always True. When the type is mutable (list), you must define it in __init__. This isn't about class attributes or shared instance attributes/constants. This is about a small optimization in defining per-instance variables. This optimization only applies to immutable types. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Style question - defining immutable class data members
On Sat, Mar 14, 2009 at 4:31 PM, Gary Herron wrote: >> Perhaps a different example would help explain what I'm trying to do: >> >> class Case1(object): >> def __init__(self): >> self.count = 0 >> self.list = [] >> >> def inc(self): >> self.count += 1 >> self.list.append(self.count) >> >> def val(self): >> return (self.count, self.list) >> >> class Case2(object): >> count = 0 >> list = [] >> >> def inc(self): >> self.count += 1 >> self.list.append(self.count) >> >> def val(self): >> return (self.count, self.list) >> >> for i in xrange(10): >> c1 = Case1() >> c2 = Case2() >> >> for j in xrange(i): >> c1.inc() >> c2.inc() >> >> v1, l1 = c1.val() >> v2, l2 = c2.val() >> >> print v1 == v2, l1 == l2 >> >> The only difference between Case1 and Case2 classes is where the count >> and list attributes are defined. You will notice that for an immutable >> type (count), this doesn't matter. On the last line, v1 == v2 is >> always True. When the type is mutable (list), you must define it in >> __init__. This isn't about class attributes or shared instance >> attributes/constants. This is about a small optimization in defining >> per-instance variables. This optimization only applies to immutable >> types. >> >> - Max >> -- >> http://mail.python.org/mailman/listinfo/python-list >> > > But now you are not listening to what people are telling you. It has > *nothing* to do with the mutability/immutability of the integer and the list > your two classes create. > > The difference is this: > > For C1: You create 10 instances of C1. Each one creates its own count, > and a list variables, and manipulates them calls to inc and val. Then each > on is discarded as you go through the next pass on the outer loop. Correct, though the discarded part makes no difference. > For C2; You create 10 instances of C2, but these 10 instances each > manipulate values created once in the class itself. The values manipulated > by one instance of C2 in one pass through the loop are not affected when, on > the next pass through the loop, that instance is destroyed and another > instance is created. > So... Incorrect. Only the count is unaffected, which was the whole point of my question. > If you want a variable that records/supplies some value across *all* > instances of a class, use a class variable. (Or use a global variable -- it > would have the same effect.) > > If you want a variable whose value is unique to each instance of a class, > then make it an instance variable. > > Gary Herron I never thought that such simple question would turn into this. David Stanek gave me the answer I was looking for (thank you). You, on the other hand, are still going after the wrong issue. Once again, here's the same example using Terry's suggestions. No class instances are being destroyed until the very end. class Case1(object): def __init__(self): self.count = 0 self.list = [] def inc(self): self.count += 1 self.list.append(self.count) def val(self): return (self.count, self.list) class Case2(object): count = 0 list = [] def inc(self): self.count += 1 self.list.append(self.count) def val(self): return (self.count, self.list) c1a, c1b = Case1(), Case1() c2a, c2b = Case2(), Case2() c1a.inc(), c1b.inc() c2a.inc(), c2b.inc() print c1a.val(), c1b.val(), c2a.val(), c2b.val() And the output: (1, [1]), (1, [1]), (1, [1, 1]), (1, [1, 1]) The first element of every tuple is the same. This is the count, which is immutable. The second element is not the same for c2[a,b]. This is the list, which is mutable. My question was about immutable values, and for those the who cases are identical. In the second case, c2a and c2b begin with count referring to the same '0' object. This is where the time and space savings are made. But because that object is immutable, when += 1 operation is performed, a copy is made for each instance. At that point, I am not sharing any values between class instances. The whole point is that this is a quick(er) way of providing initial values for class instance variables when those values are immutable. When/if that initial value is changed, a copy is made. In Case1, that copy is made from the very begging in __init__. Please try to understand what the question is about before responding to it. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Style question - defining immutable class data members
On Sat, Mar 14, 2009 at 5:38 PM, Matthew Woodcraft wrote: > Gary Herron writes: > I think this code is in poor taste: it's clear that it will confuse > people (which is what Maxim was asking about in the first place). Yes, I see that now, thanks :) - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Script for a project inside own directory
2009/3/22 Filip Gruszczyński :
> I am having a project built like this:
>
> project
> module1.py
> module2.py
> packages1/
> module3.py
>
> etc.
>
> I have script that uses objects from those modules/packages. If I keep
> this script inside project directory it's ok and it works. But I would
> like to move it to own scripts directory and from there it doesn't
> work. I think I understand why it doesn't work (the root of local
> packages and modules is there and it can't see what it above it), but
> I would like to ask if there is any workaround? I would like to keep
> all my scripts in separate dir instead of main dir. I like to keep it
> clean.
import sys
sys.path.append('')
If project directory is one level up, you can do something like this:
import os
import sys
sys.path.append(os.path.realpath('..'))
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: How Get Name Of Working File
On Sun, Mar 22, 2009 at 10:58 AM, Christian Heimes wrote: > Victor Subervi schrieb: >> Hi; >> If I am writing a script that generates HTML, how do I grab the name of the >> actual file in which I am working? For example, let us say I am working in >> test.py. I can have the following code: >> >> import os >> dir = os.getcwd() >> >> and that will give me the working dir. But what about "test.py"? > > The module variable __file__ contains the file name of the current > Python module. Keep in mind that __file__ may be set to test.pyc or test.pyo. If you always want the .py extension, do this: from os.path import splitext file = splitext(__file__)[0] + '.py' - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 10:16 AM, CinnamonDonkey wrote: > Hi All, > > I'm fairly new to Python so I still have a lot to learn. But I'd like > to know how to correectly use relative imports. > > Please, please... please! don't go off on rants about why you think > relative imports should not be used. I've got 15+ years in C++ and > relative inclusion of other sections of code has never been a problem. > As far as I am concerned what I am trying to do is perfectly > reasonable and valid. > > Thank you in advance to everyone who helps solve this, because I just > don't get it. > > Example: > > \ App > | main.py > +--\subpack1 > | | __init__.py > | | module1.py > | > +--\subpack2 > | | __init__.py > | | module2.py > > > Module1 needs to access functionality in Module2. > > #module1.py > from ..subpack2 import module2 > > Seems reasonable to me... but it just does not work and I was so > liking Python. :( Relative imports are perfectly fine, in my opinion. Do you have "from __future__ import absolute_import" at the top of module1.py? Should work fine once you add that line. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 11:22 AM, CinnamonDonkey wrote: > Looking at http://www.python.org/dev/peps/pep-0328/#guido-s-decision > would suggest, unless I am completely miss-understanding the example, > that '.' refers to the current level and '..' pops up a level. That is correct, but you cannot jump beyond the parent package, which is why your code isn't working. > Max, thank you for the response... I tried adding "from __future__ > import absolute_import" which made no difference. I still get exactly > the same error messages. Perhaps I should have mentioned that I am > using Python 2.5, which I understand alread supports relative imports > out of the box. I'll keep this line in for now anyway though :-) > Cheers! Sorry, I use that line to avoid conflicts with standard modules, and forgot that relative imports are already enabled. Basically, the reason your code doesn't work is because you're trying to use relative imports between two separate packages. As far as I know, this isn't possible. What I did to get your code working was move main.py one directory up and create an empty __init__.py under \App. The following code should then work: # main.py import App.subpack1.module1 if __name__ == "__main__": App.subpack1.module1.subpack1_module1_foo() # App.subpack1.module1 from ..subpack2 import module2 def subpack1_module1_foo(): print "subpack1_module1_foo()" call_subpack2_module1() def call_subpack2_module1(): module2.subpack2_module2_foo() # App.subpack2.module2 def subpack2_module2_foo(): print "subpack2_module2_foo()" - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 12:19 PM, CinnamonDonkey wrote: > My applogies if this is a silly question... but what makes something a > package? and does that mean that what I am trying to do is not > possible ? A package is a directory that has an __init__.py file. That file can be empty, or contain some initialization code. In your original example, subpack1 and subpack2 are packages. By adding an empty __init__.py file under \App, I made App into a package, which allowed me to execute "import App.subpack1.module1" in main.py. See the following url for additional info: http://docs.python.org/tutorial/modules.html - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Tue, Mar 24, 2009 at 5:05 AM, CinnamonDonkey
wrote:
> Thanx Max - your explanation sorted it :-), and a big thank you to
> everyone else also!
>
> >From the various posts, Python considers any directory containing the
> __init__.py file to be a package. The top level package is the highest
> directory (closest to root) with a __init__.py file.
>
> Inter-package communication is not allowed unless the packages
> themselves are contained by a parent package.
>
> How does this relate to the site-packages folder? Is it a top level
> package for all installed packages?
>
> Let's say I have installed the "Trac" system which uses "Genshi", they
> are both packages. They are also both installed at the same level and
> I know "Trac" uses "Genshi" to work. \Python25\Lib\site-packages does
> not contain a __init__.py file so it is not a package (i.e. not a
> parent package to "Trac" and "Genshi") :0.
Trac does not use relative imports to access genshi. When relative
imports are not used, python goes through sys.path list to find
modules (with a small exception made when absolute_imports are not
enabled, but that should be default in 2.7). The site-packages
directory is added to sys.path, so when trac executes something like
"from genshi import some_module", python will look in site-packages,
among other directories, for a directory called "genshi" that contains
an __init__.py file.
When you execute a script, the directory of that script is
automatically added to sys.path, so with your example you could have
used absolute imports between subpack1 and subpack2, with the \App
directory performing the same function as site-packages (Gabriel's
suggestion). This is for your original version of the code when
main.py was under App.
Once you moved main.py outside of \App, running "import sybpack2"
would no longer work. You can, however, append directories to
sys.path, so by doing the following in main.py you could again allow
non-relative imports between subpack1 and subpack2:
import os
import sys
sys.path.append(os.path.realpath('App'))
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unit testing frameworks
On Tue, Mar 24, 2009 at 8:06 AM, wrote:
> I am looking for a unit testing framework for Python. I am aware of
> nose, but was wondering if there are any others that will
> automatically find and run all tests under a directory hierarchy.
Have you already looked at the unittest module? Below is the code I
use for one of my current projects to load all test cases in package.
This code is sitting in __init__.py, and the test cases are in
separate files (util.py, util_threading.py, etc.). Those files can
contain as many TestCase classes as needed, all are loaded with
loadTestsFromModule. You could easily modify this code to
automatically generate the modules list if you want to.
# repo/pypaq/test/__init__.py
from unittest import TestSuite, defaultTestLoader
import logging
import sys
__all__ = ['all_tests']
modules = ['util', 'util_buffer', 'util_event', 'util_threading']
if not __debug__:
raise RuntimeError('test suite must be executed in debug mode')
all_tests = []
for name in modules:
module = __import__('pypaq.test', globals(), locals(), [name], 0)
tests = defaultTestLoader.loadTestsFromModule(getattr(module, name))
__all__.append(name)
all_tests.append(tests)
setattr(sys.modules[__name__], name, tests)
logging.getLogger().setLevel(logging.INFO)
all_tests = TestSuite(all_tests)
I then have test_pypaq.py file under repo/, with which I can execute
all_tests or only the tests from a specific module:
# repo/test_pypaq.py
from unittest import TextTestRunner
from pypaq.test import *
TextTestRunner(verbosity=2).run(all_tests)
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Tue, Mar 24, 2009 at 8:57 PM, Istvan Albert wrote: > Does it not bother you that a module that uses relative imports cannot > be run on its own anymore? $ python --help -m mod : run library module as a script (terminates option list) $ python -m some.module.name Works perfectly fine with relative imports. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: OpenGL win32 Python
On Fri, Apr 17, 2009 at 9:27 AM, gintare statkute wrote: > Hello, > > i found an example for OpenGL in windows. > It is incredibly helpful, but how to rewrite it to be useful in Python. > > How to give address of pfd in Python?: > iFormat = ChoosePixelFormat( hDC, &pfd ); > SetPixelFormat( hDC, iFormat, &pfd ); Take a look at ctypes.byref and ctypes.pointer. Post the results if you get it to work :) - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Efficient bits manipulation in Python
On Tue, Apr 28, 2009 at 7:26 AM, Li Wang wrote: > Hi: > > I have a bit-code :'1011011', how can I reverse it to '1101101'? > > Another question is I know how to transform the string '110' into > integer 6, does anyone know how to transform integer 6 to a string > '110'? > > Thank you very much:) Assuming that you are using 2.6: a = 0b1011011 print bin(a)[:1:-1] a = 6 print bin(a)[2:] - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: How to timeout when waiting for raw_input from user ?
On Fri, Dec 4, 2009 at 6:55 PM, northof40 wrote:
> On Dec 5, 12:52 pm, northof40 wrote:
>> Hi - I'm writing a *very* simple program for my kids. It asks the user
>> to give it the answer to a maths question and says "right" or "wrong"
>>
>> They now want a timed version where they would only get so long to
>> respond to the question.
>>
>> I'm thinking of some logic where a raw_input call is executed and then
>> if more than X seconds elapses before the prompt is replied to the
>> process writes a message "Sorry too slow" (or similar).
>>
>> I can't see the wood for the trees here - what's the best way to do
>> this given the rather simple environment it's needed within.
>>
>> Regards
>>
>> richard.
>
> Sorry I should said that based upon other answers I've seen to similar
> questions this needs to run on a windows machine (other answers
> suggest this is more difficult than running on *nix)
>
Simplest solution I could come up with. This is indeed much easier on
*nix (just use select.select on sys.stdin with a timeout).
---
from msvcrt import getch, kbhit, putch
from time import sleep, time
ans = ''
end = time() + 5
print('2 + 2 = ?')
while True:
while time() < end:
if kbhit():
break
else:
sleep(0.001)
else:
ans = None
break
char = getch()
if char == '\r':
print('')
break
ans += char
putch(char)
if ans is None:
print('\nSorry too slow')
else:
try:
print('right' if int(ans) == 4 else 'wrong')
except:
print('not a number')
---
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to timeout when waiting for raw_input from user ?
On Sat, Dec 5, 2009 at 9:01 AM, Rune Strand wrote: > The easiest wasy is to use the Timer object in the threading module. > > > from threading import Timer Doesn't work on Windows. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: How to timeout when waiting for raw_input from user ?
On Sat, Dec 5, 2009 at 9:11 AM, Rune Strand wrote: > On Dec 5, 3:07 pm, Maxim Khitrov wrote: >> >> Doesn't work on Windows. >> >> - Max > > Yes, it does. I've used it a lot, also in Py2Exe apps. Try the > documentation example yourself > > def hello(): > print "hello, world" > > t = Timer(30.0, hello) > t.start() # after 30 seconds, "hello, world" will be printed I'm not talking about the Timer, I'm talking about the original question. There's nothing (that I know of) you can do with a Timer on Windows to interrupt a raw_input call. - Max -- http://mail.python.org/mailman/listinfo/python-list
Internationalized To:'s and Cc:'s
Hello! How should I correctly construct internationalized base64'ed MIME header? The problem is that 'real name' _should_ be encoded, but the email address - should not. For example, ?utf-8?bla-bla=?= should be the correct format, and ?utf-8?bla-bla-bla-bla=?= - incorrect. But email.Header encodes full text including address, and then local mail server append's local domain name as it is not found in encoded header in plain form. I could manually base64 encode real name, append address, put it in header without encoding, but is there any more suitable way doing it? (telling Header to leave address part of header un-encoded). -- http://mail.python.org/mailman/listinfo/python-list
Inconsistent raw_input behavior after Ctrl-C
Hello all, I ran into a rather strange problem when interrupting a raw_input call with Ctrl-C. This is with python 2.6.3 on Windows 7. When the call is interrupted, one of two things happen - either a KeyboardInterrupt exception is raised or raw_input raises EOFError, and KeyboardInterrupt is raised a line or two later. Here's the example that I'm testing with: import sys import traceback print '===' excs = [] try: try: raw_input() except BaseException as exc: excs.append(sys.exc_info()) print '1', type(exc) except BaseException as exc: excs.append(sys.exc_info()) print '2', type(exc) print '---' for exc in excs: traceback.print_exception(*exc) print '===' And here are all the two different outputs that I've received at random times: === 1 --- Traceback (most recent call last): File "client.py", line 26, in raw_input() KeyboardInterrupt === === 1 2 --- Traceback (most recent call last): File "client.py", line 26, in raw_input() EOFError Traceback (most recent call last): File "client.py", line 29, in print '1', type(exc) KeyboardInterrupt === This makes no sense to me. First, where does raw_input get EOF (Ctrl-Z or F6) from? Second, why is KeyboardInterrupt raised in the middle of executing a print instruction and not at raw_input? Third, if the inner except clause finishes too soon (for example, if I comment out the print statement), then the KeyboardInterrupt is sometimes raised at the print '---' line. This makes it difficult to consistently handle a Ctrl-C event without calling something like sleep after an EOFError. I don't recall seeing this problem in Windows XP, but I'm not able to test on it right now. Is this problem related to Windows 7 in some way? - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Inconsistent raw_input behavior after Ctrl-C
On Tue, Oct 20, 2009 at 6:09 PM, Maxim Khitrov wrote: > Hello all, > > I ran into a rather strange problem when interrupting a raw_input call > with Ctrl-C. This is with python 2.6.3 on Windows 7. When the call is > interrupted, one of two things happen - either a KeyboardInterrupt > exception is raised or raw_input raises EOFError, and > KeyboardInterrupt is raised a line or two later. > > This makes no sense to me. First, where does raw_input get EOF (Ctrl-Z > or F6) from? Second, why is KeyboardInterrupt raised in the middle of > executing a print instruction and not at raw_input? Third, if the > inner except clause finishes too soon (for example, if I comment out > the print statement), then the KeyboardInterrupt is sometimes raised > at the print '---' line. This makes it difficult to consistently > handle a Ctrl-C event without calling something like sleep after an > EOFError. > > I don't recall seeing this problem in Windows XP, but I'm not able to > test on it right now. Is this problem related to Windows 7 in some > way? > > - Max > Replying to my own post... The code below seems to fix the problem, though it is obviously a hack I would rather do without. I have no idea if 50ms is enough to wait for a KeyboardInterrupt, but so far I've not encountered any inconsistent behavior. import __builtin__ import time def raw_input2(prompt=''): """ Workaround for raw_input raising EOFError and KeyboardInterrupt on Ctrl-C. """ try: return raw_input1(prompt) except EOFError as exc: # If KeyboardInterrupt is not raised in 50ms, it's a real EOF event. time.sleep(0.05) raise raw_input1 = raw_input __builtin__.raw_input = raw_input2 try: raw_input() except BaseException as exc: print type(exc) -- http://mail.python.org/mailman/listinfo/python-list
