Re: using python to parse md5sum list
On 5 Mar 2005 19:54:34 -0800, rumours say that [EMAIL PROTECTED] (Ben Rf) might have written: [snip] >the end end goal is to have a way of finding duplicate files that are >scattered across a lan of 4 windows computers. Just in case you want to go directly to that goal, check this: http://groups-beta.google.com/group/comp.lang.python/messages/048e292ec9adb82d It doesn't read a file at all, unless there is a need to do that. For example, if you have ten small files and one large one, the large one will not be read (since no other files would be found with the same size). In your case, you can use the find_duplicate_files function with arguments like: r"\\COMPUTER1\SHARE1", r"\\COMPUTER2\SHARE2" etc -- TZOTZIOY, I speak England very best. "Be strict when sending and tolerant when receiving." (from RFC1958) I really should keep that in mind when talking with people, actually... -- http://mail.python.org/mailman/listinfo/python-list
HELP: Python & Tkinter ?
Hi all, Have just installed Fedora 3 and wish also want to give programming with Python a go. I wish to develop GUI interfaces and believe that 'Tkinter' is the way to go. After installing Python frpm the distro CD I can't use the 'from Tkinter import *' command. When I try, I get the error message below: ImportError: No module named Tkinter So from that I assume I need to install Tkinter ? Should I also find this on the CD ? Is it known as another name as looking on the CD I don't seem to find anything relating to Tkinter. Thanks in advance. Pete -- http://mail.python.org/mailman/listinfo/python-list
Re: IndexedCatalog and ZEO
Leif K-Brooks wrote:
> from IndexedCatalog.Shelf import Shelf
> shelf = Shelf(('localhost', 1234), [Class1, Class2])
Thanks, that is what I was searching for.
--
Lukas "Almad" Linhart
[:: http://www.almad.net/ ::]
[:: Humans are too complicated to be described with words. ::]
[:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python & Tkinter ?
"PGMoscatt" wrote: > Have just installed Fedora 3 and wish also want to give programming with > Python a go. I wish to develop GUI interfaces and believe that 'Tkinter' > is the way to go. > > After installing Python frpm the distro CD I can't use the 'from Tkinter > import *' command. When I try, I get the error message below: > > ImportError: No module named Tkinter > > So from that I assume I need to install Tkinter ? > Should I also find this on the CD ? Is it known as another name as looking > on the CD I don't seem to find anything relating to Tkinter. according to http://tkinter.unpythonic.net/wiki/How_20to_20install_20Tkinter this should do the trick: yum install tkinter -- http://mail.python.org/mailman/listinfo/python-list
Re: HELP: Python & Tkinter ?
On Mon, 07 Mar 2005 18:19:16 +1000, rumours say that PGMoscatt <[EMAIL PROTECTED]> might have written: >Hi all, > >Have just installed Fedora 3 and wish also want to give programming with >Python a go. I wish to develop GUI interfaces and believe that 'Tkinter' >is the way to go. [snip: import Tkinter fails, probably due to missing package] Perhaps this page helps: http://www.python.org/2.4/rpms.html Otherwise this would show up anything in the cd: $ find /mnt/cdrom -name \*kinter\* -print (/mnt/cdrom or whatever the name of the cd mounting directory in Fedora 3 is) -- TZOTZIOY, I speak England very best. "Be strict when sending and tolerant when receiving." (from RFC1958) I really should keep that in mind when talking with people, actually... -- http://mail.python.org/mailman/listinfo/python-list
Mail System Error - Returned Mail
The original message was included as attachment test; hi; hello; Mail Delivery System; Mail Transaction Failed; Server Report; Status; Error; Test; Hi; Hello; Encrypted Mail; Virus sample; abuse?; feel free to use it; Excel file; Details; fake; read it immediately; something for you; information; order; encrypted document; file is bad; your document; your archive; re: unknow; re: questions; report; is that your account?; re: protected message; hidden message; Mail Delivery; failure notice; Picture Size: 11 KB, Mail: +OK; Fw: Buon Natale!-- http://mail.python.org/mailman/listinfo/python-list
Re: How do I import everything in a subdir?
Hello,
If you want to look for the files "*.py" in a directory, don't use
shell command!!!
You have many ways to access the content of a directory in Python.
For exemple, you can use the glob module:
>>> import glob
>>> glob.glob('./[0-9].*')
['./1.gif', './2.txt']
>>> glob.glob('*.gif')
['1.gif', 'card.gif']
>>> glob.glob('?.gif')
['1.gif']
You might look at this page "http://docs.python.org/lib/module-glob.html";.
Cyril
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python & Tkinter ?
Fredrik Lundh wrote: > "PGMoscatt" wrote: > >> Have just installed Fedora 3 and wish also want to give programming with >> Python a go. I wish to develop GUI interfaces and believe that 'Tkinter' >> is the way to go. >> >> After installing Python frpm the distro CD I can't use the 'from Tkinter >> import *' command. When I try, I get the error message below: >> >> ImportError: No module named Tkinter >> >> So from that I assume I need to install Tkinter ? >> Should I also find this on the CD ? Is it known as another name as >> looking on the CD I don't seem to find anything relating to Tkinter. > > according to > > http://tkinter.unpythonic.net/wiki/How_20to_20install_20Tkinter > > this should do the trick: > > yum install tkinter > > Thanks Fredrik, yep, that did the trick. Never seen this YUM thing but it's pretty trick. Pete -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible to import a module whose name is contained in a variable?
STeVe, may I ask you for more details on this? Any disadvantages while using exec() in this context? I try to avoid using any of the ____() functions if possible (considering this a good programming style). Claudio "Steven Bethard" <[EMAIL PROTECTED]> schrieb im Newsbeitrag news:[EMAIL PROTECTED] > Claudio Grondi wrote: > > "Steven Reddie" <[EMAIL PROTECTED]> schrieb im Newsbeitrag > > news:[EMAIL PROTECTED] > >> > >>I want to do something like the following, which doesn't work: > >> > >>modulename = 'module' > >>import modulename > >> > >>The error is that there is no module named 'modulename'. Is there a > >>way to get that variable expanded? > > > > modulename = 'module' > > cmd = 'import '+modulename > > exec(cmd) > > > > Check also the thread: > > How do I import everything in a subdir? > > in THIS newsgroup. > > And note that it tells you to use __import__, not exec. =) > > STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible to import a module whose name is contained in a variable?
On Monday 07 March 2005 11:52, Claudio Grondi wrote:
> I try to avoid using any of the
> ____() functions if possible
> (considering this a good
> programming style).
This is never good style, at least in the case of exec. exec is evil.
What works (beware that the below code is nevertheless untested and might
contain little warts) and is the "usual" and clean way to go:
### libinfo/__init__.py
# Empty.
### libinfo/Module1.py
def libinfo():
return "I am module1."
CFLAGS = ["-DMODULE1"]
### libinfo/Module2.py
def libinfo():
return "I am module2."
CFLAGS = ["-DMODULE2"]
### Importer.py
modules = {}
CFLAGS = []
def load_modules(to_load=["module1","module2"]):
global modules, CFLAGS
for mod in to_load:
try:
modules[mod] = getattr(__import__("libinfo.%s" % mod),mod)
except ImportError:
print "Could not load %s." % mod
continue
print "Module: %s (%r)." % (mod,modules[mod])
print "Modules libinfo: %r." % modules[mod].libinfo()
CFLAGS += modules[mod].CFLAGS
print "Total CFLAGS: %s." % CFLAGS
if __name__ == "__main__":
load_modules()
print "Module container: %s." % modules
### End Importer.py
HTH!
--
--- Heiko.
pgpfjzzhE3U5B.pgp
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
mxDateTime on Mac: Fatal Python Error
I'm trying to get mxDateTime working on a Mac so that I can use pyscopg and cx_Oracle. The Egenix base package builds and installs quite happily, but then when I try to import it I get >> import mx.DateTime Fatal Python error: Interpreter not initialized (version mismatch?) Abort ... any ideas? Environment: OS X 10.3.8 sys.version: '2.3 (#1, Sep 13 2003, 00:49:11) \n[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)]' egenix-mx-base-2.0.6 -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible to import a module whose name is contained in a variable?
Claudio Grondi wrote:
STeVe,
may I ask you for more details on this?
Any disadvantages while using exec()
in this context?
I try to avoid using any of the
____() functions if possible
(considering this a good
programming style).
Avoiding exec (which is a statement, not a function) is much more
important. Since it executes arbitrary code, you can get unpredictable
results from it.
>>> z = "sys; print 'w00t'"
>>> exec "import " + z
w00t
Consider the case where z = "shutil; shutil.rmtree('/')"
--
Michael Hoffman
--
http://mail.python.org/mailman/listinfo/python-list
Re: GOTO (was Re: Appeal for python developers)
On Sunday 06 March 2005 14:26, Anthra Norell wrote:
>
Wow, I never thought I'd say this, but this certainly is an ingenious use of
goto... But, nevertheless, I don't think this is applicable to Python as a
way of justifying goto in the language, as your program doesn't have a split
between abstract state machine and real program anymore (which I think should
be there, as the abstract state machine is actually data, which controls
program flow).
The way I'd code it in Python is something like:
SETUP = object()
ELSE = object()
BREAK = object()
machine = {"WAITING FOR ACTION":
{customer_drops_coin:"COIN HAS BEEN DROPPED",
customer_selects_beverage:"ORDER RECEIVED",
customer_cancels_order:"ACCOUNT CLOSURE IS DUE"
ELSE:"WAITING FOR ACTION"},
"COIN HAS BEEN DROPPED":
{SETUP:identify_coin,
credit_account:"PAYMENT DUE IS UNKNOWN"},
"ORDER RECEIVED":
{...
Reading the state machine in the way presented above isn't any harder in my
taste than reading your state table, and you should easily be able to run the
machine from there...
--
--- Heiko.
pgpFSqfb8wW7l.pgp
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
win32com and Python2.4: Interpreter crashing!
Hi, When trying to use win32com functionality with Python2.4, the interpreter often crashes! I don't know if it's a known problem already that will be fixed in an upcoming Python2.4.1 version. Basically, the problem is: makepy.py generates a (large, nearly 2mb) Python file for use with COM interfaces of an MS Excel. Generating the file seems to work fine. However, when trying to import this file, the interpreter crashes trying to compile it into a .pyc file. The same file gives no such problems with Python 2.3.5. I first just used makepy.py with Python2.4, but it crashed after generating a large Python file. Then I created the file using python2.3.5, and copied it to my python2.4 installation, and that crashes python2.4 when importing it. So as I said before: is this a known problem already fixed for the next release, or should I collect crash-information for this problem? cheers, --Tim -- http://mail.python.org/mailman/listinfo/python-list
re - Question about pyFMOD importing
Dear Tian, perhaps you might be able to assit me. I've seen your question regarding pyMod importing http://mail.python.org/pipermail/python-list/2005-February/263969.html I'm using a different module (something to do with paralell port) but after installing my modules I get the same erro as yours Were you able to hack this one ? Best regards Lior Botzer -- http://mail.python.org/mailman/listinfo/python-list
Re: Modifying Call Tips and Intellisense Behavior
have a look at eclipse + pyDev http://pydev.sourceforge.net/ probably it works as you wish -- http://mail.python.org/mailman/listinfo/python-list
resources
hi i'v been searching 4 resources ,and study materials on PYTHON.If any body has some suggestion for getting ebooks ,pdf,or tutorials kindly let me know. thankyou -- http://mail.python.org/mailman/listinfo/python-list
reversed heapification?
Hi! I need a general-purpose best-k sorting algorithm and I'd like to use heapq for that (heapify + heappop*k). The problem is that heapify and heappop do not support the "reverse" keyword as known from sorted() and list.sort(). While the decorate-sort-undecorate pattern allows me to replace the "key" option, I do not see an obvious way to have heapq work in a reverse way without making assumptions on the data. Any ideas? Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: reversed heapification?
Stefan Behnel wrote: Hi! I need a general-purpose best-k sorting algorithm and I'd like to use heapq for that (heapify + heappop*k). The problem is that heapify and heappop do not support the "reverse" keyword as known from sorted() and list.sort(). While the decorate-sort-undecorate pattern allows me to replace the "key" option, I do not see an obvious way to have heapq work in a reverse way without making assumptions on the data. heapq.nlargest() heapq.nsmallest() ? Python 2.4 only Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: reversed heapification?
Kent Johnson schrieb: heapq.nlargest() heapq.nsmallest() ? Python 2.4 only Thanks! Those are *very* well hidden in the documentation. Maybe I already read that page too often... Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: re - Question about pyFMOD importing
On Tuesday 08 March 2005 12:38, lior botzer wrote: > Were you able to hack this one ? I haven't seen this error in a long time (as I'm no Windows user for a long time), but from what I gather the only thing that the specified error was telling you is the fact that the dynamic linking library that ctypes wanted to load wasn't there. I guess that the library that ctypes looks for is the DLL required for FMOD (in the original example). So, basically, what I can tell you: to remedy this error, put the required DLL (whose name you can find in the sources) somewhere on the search path, so that it can be loaded by the C-call LoadLibrary(), which ctypes just conveniently wraps. The library search path should include the current path (along with %WINDIR%/system and %WINDIR%/system32), so putting the DLL there should work too. -- --- Heiko. pgpeyJhCNG6uT.pgp Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
os.system()
Hello, code like os.system(command) only works for some values of 'command' on my system (Linux). A certain shell command (that *does* run on the command line) does not work when called with os.system(). Does anyone know a simple and stable way to have *any* string executed by the shell? Jörg Schuster -- http://mail.python.org/mailman/listinfo/python-list
Re: reversed heapification?
Kent Johnson wrote: heapq.nlargest() heapq.nsmallest() On second thought, that doesn't actually get me very far. I do not know in advance how many I must select since I need to remove duplicates *after* sorting (they are not necessarily 'duplicate' enough to fall into the same sort bucket). What I'd like to do is heapify and then create an iterator for the result. But since heapify doesn't support "reverse" ... Any other ideas? Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
> only works for some values of 'command' on my system (Linux). A certain > shell command (that *does* run on the command line) does not work when > called with os.system(). Does anyone know a simple and stable way to > have *any* string executed by the shell? Showing us what commands actually fail would certainly help. -- Regards, Diez B. Roggisch -- http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
Well, I can give you the string, but that will not help: transduce abc info_dic comp_dic input_file output_file For copy right reasons, I am not allowed to give you the program transduce. But here are some facts about transduce: - it is written in C - it takes an alphabet file (abc) and two automata files (info_dic and comp_dic) as input and applies the automata files to the input file. -- http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
On Monday 07 March 2005 14:10, Diez B. Roggisch wrote:
> Showing us what commands actually fail would certainly help.
Actually, this sounds like the subshell isn't getting an alias that the normal
interactive shell has. Maybe because ~/.bashrc isn't read on os.system(), or
something of the like? This depends largely on your default system settings,
and especially on /etc/profile.
You might check whether the command that works in the interactive shell is an
alias by typing
[EMAIL PROTECTED] ~ $ alias
alias ls='ls --color=auto'
[EMAIL PROTECTED] ~ $
This shows all currently set aliases, and at least on Gentoo, the above alias
is set in ~/.bashrc, and thus isn't set when os.system() is called. This
means that the output from running ls in an interactive shell is colorized,
whereas running os.system("ls") from Python is not colorized, although
TERM="xterm" in os.environ, and thusly in the subshell spawned using
os.system, and ls could colorize the output using VT100 escape sequences.
All the above explanations assume that your default shell /bin/sh is the
Bourne Again Shell, but all other "higher shells" such as the (T)C-Shell and
the Korn-Shell support command aliasing too, in some way or another, and will
suffer from the same quirks.
And, btw., it'll help if you read the commented start-up files (at least on
Gentoo and SuSE (IIRC) they are very well commented) and the bash man-page,
they explain pretty clearly which initialization files (~/.bashrc,
~/.bash_profile, /etc/profile, /etc/bash/bashrc, and several others) get
executed when and where, depending on whether a shell is a login shell (your
normal interactive shell), or not (spawned by os.system, for example).
Hope this explanation helps!
--
--- Heiko.
pgpEIm4nKNAgd.pgp
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible to import a module whose name is contained in a variable?
On Mar 7, 2005, at 5:23 AM, Michael Hoffman wrote:
Avoiding exec (which is a statement, not a function) is much more
important. Since it executes arbitrary code, you can get unpredictable
results from it.
Is there any way to use __import__ to replace the following:
exec("from %s import *" % modulename)
___/
/
__/
/
/
Ed Leafe
http://leafe.com/
http://dabodev.com/
Come to PyCon http://www.python.org/pycon/2005/
--
http://mail.python.org/mailman/listinfo/python-list
PyAr - Python Argentina 7th Meeting, this Thursday
Title: PyAr - Python Argentina 7th Meeting, this Thursday The Argentine Python User Group, PyAr, will have its seventh meeting this Thursday, January 10th at 7:00pm. Agenda -- Despite our agenda tends to be rather open, this time we would like to cover these topics: - See how the code evolved from the ideas generated in of our first sprint. - Define how we will promote ourselves in PyCon 2005. Where - We're meeting at Hip Hop Bar, Hipólito Yirigoyen 640, Ciudad de Buenos Aires, starting at 19hs. We will be in the back room, so please ask the barman for us. About PyAr -- For more information on PyAr see http://pyar.decode.com.ar (in Spanish), or join our mailing list (Also in Spanish. For instructions see http://pyar.decode.com.ar/Members/ltorre/listademail) We meet on the second Thursday of every month. . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA. La información contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener información confidencial o propietaria, cuya divulgación es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no está autorizado a divulgar, copiar, distribuir o retener información (o parte de ella) contenida en este mensaje. Por favor notifíquenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magnético) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telefónica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electrónicos pueden ser alterados, motivo por el cual Telefónica Comunicaciones Personales S.A. no aceptará ninguna obligación cualquiera sea el resultante de este mensaje. Muchas Gracias. -- http://mail.python.org/mailman/listinfo/python-list
Re: convert gb18030 to utf16
Truely superb!
Thanks!
Xah
[EMAIL PROTECTED]
http://xahlee.org/
[EMAIL PROTECTED] wrote:
> Xah Lee <[EMAIL PROTECTED]> wrotE:
>
> > i have a bunch of files encoded in GB18030. Is there a way to
convert
> > them to utf16 with python?
>
> You will need CJKCodecs (http://cjkpython.i18n.org/), or Python 2.4,
> which has them built in. Then just use them like any other codec. eg.
>
> f= open(path, 'rb')
> content= unicode(f.read(), 'gb18030')
> f.close()
> f= open(path, 'wb')
> f.write(content.encode('utf-16'))
> f.close()
>
> --
> Andrew Clover
> mailto:[EMAIL PROTECTED]
> http://www.doxdesk.com/
--
http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
> Several variables like PATH "normally" get reset even when > running a non-login subshell It seems that this has been the problem. I guess your tip saved me a lot of time. Thanks a lot. Joerg -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible to import a module whose name is contained in a variable?
Ed Leafe wrote:
> On Mar 7, 2005, at 5:23 AM, Michael Hoffman wrote:
> >Avoiding exec (which is a statement, not a function) is much more
> >important. Since it executes arbitrary code, you can get unpredictable
> >results from it.
>
> Is there any way to use __import__ to replace the following:
>
> exec("from %s import *" % modulename)
No.
Gerrit.
--
Weather in Twenthe, Netherlands 07/03 13:25:
4.0ÂC Few clouds mostly cloudy wind 5.4 m/s NW (57 m above NAP)
--
In the councils of government, we must guard against the acquisition of
unwarranted influence, whether sought or unsought, by the
military-industrial complex. The potential for the disastrous rise of
misplaced power exists and will persist.
-Dwight David Eisenhower, January 17, 1961
--
http://mail.python.org/mailman/listinfo/python-list
Re: reversed heapification?
Stefan Behnel wrote: Kent Johnson wrote: heapq.nlargest() heapq.nsmallest() On second thought, that doesn't actually get me very far. I do not know in advance how many I must select since I need to remove duplicates *after* sorting (they are not necessarily 'duplicate' enough to fall into the same sort bucket). What I'd like to do is heapify and then create an iterator for the result. But since heapify doesn't support "reverse" ... Any other ideas? Wrap your data in a class that defines __cmp__ as the inverse of __cmp__ on the underlying data, then use heapq? Just sort the list? Kent Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
> Several variables like PATH "normally" get reset even when > running a non-login subshell It seems that this has been the problem. I guess your tip saved me a lot of time. Thanks a lot. Joerg -- http://mail.python.org/mailman/listinfo/python-list
Re: Unable to run IDLE Under Windows
Thanks, guess I misunderstood--I thought "pythonw" _was_ IDLE. Now I see what IDLE is, and I wasn't actually wanting to run that. And as it turns out, my _real_ problem was that my path was making me run the CygWin version of Python from the "DOS" command prompt--which understandably dies with a "CPU" error. Now I'm squared away--thanks for the tip! -- http://mail.python.org/mailman/listinfo/python-list
Re: resources
In article <[EMAIL PROTECTED]>, shield0092005 <[EMAIL PROTECTED]> wrote: > >hi >i'v been searching 4 resources ,and study materials on PYTHON.If any >body has some suggestion for getting ebooks ,pdf,or tutorials kindly >let me know. >thankyou > > > If your search did not encounter http://python.org/doc/ >--and surely you would have mentioned it if you'd found it on your own--I wonder what suggestion will best serve you. -- http://mail.python.org/mailman/listinfo/python-list
shuffle the lines of a large file
Hello, I am looking for a method to "shuffle" the lines of a large file. I have a corpus of sorted and "uniqed" English sentences that has been produced with (1): (1) sort corpus | uniq > corpus.uniq corpus.uniq is 80G large. The fact that every sentence appears only once in corpus.uniq plays an important role for the processes I use to involve my corpus in. Yet, the alphabetical order is an unwanted side effect of (1): Very often, I do not want (or rather, I do not have the computational capacities) to apply a program to all of corpus.uniq. Yet, any series of lines of corpus.uniq is obviously a very lopsided set of English sentences. So, it would be very useful to do one of the following things: - produce corpus.uniq in a such a way that it is not sorted in any way - shuffle corpus.uniq > corpus.uniq.shuffled Unfortunately, none of the machines that I may use has 80G RAM. So, using a dictionary will not help. Any ideas? Joerg Schuster -- http://mail.python.org/mailman/listinfo/python-list
Re: reversed heapification?
Can you use something like (untested) class ComparisonReverser: def __init__(self, s): self.s = s def __cmp__(self, o): return cmp(o, self.s) def __lt__... # or whichever operation hashes use then use (ComparisonReverser(f(x)), i, x) as the decorated item instead of (f(x), i, x) Jeff pgpKui90u477G.pgp Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
Joerg Schuster wrote: Hello, I am looking for a method to "shuffle" the lines of a large file. I have a corpus of sorted and "uniqed" English sentences that has been produced with (1): (1) sort corpus | uniq > corpus.uniq corpus.uniq is 80G large. The fact that every sentence appears only once in corpus.uniq plays an important role for the processes I use to involve my corpus in. Yet, the alphabetical order is an unwanted side effect of (1): Very often, I do not want (or rather, I do not have the computational capacities) to apply a program to all of corpus.uniq. Yet, any series of lines of corpus.uniq is obviously a very lopsided set of English sentences. So, it would be very useful to do one of the following things: - produce corpus.uniq in a such a way that it is not sorted in any way - shuffle corpus.uniq > corpus.uniq.shuffled Unfortunately, none of the machines that I may use has 80G RAM. So, using a dictionary will not help. There was a thread a while ago about choosing random lines from a file without reading the whole file into memory. Would that help? Instead of shuffling the file, shuffle the users. I can't find the thread though... Kent -- http://mail.python.org/mailman/listinfo/python-list
RE: shuffle the lines of a large file
Not tested this, run it (or some derivation thereof) over the output to get
increasing randomness.
You will want to keep max_buffered_lines as high as possible really I
imagine. If shuffle() is too intensize
you could itterate over the buffer several times randomly removing and
printing lines until the buffer is empty/suitibly small removing some more
processing overhead.
### START ###
import random
f = open('corpus.uniq')
buffer = []
max_buffered_lines = 1000
for line in f:
if len(buffer) < max_buffered_lines:
buffer.append(line)
else:
buffer.shuffle()
for line in buffer:
print line
random.shuffle(buffer)
for line in buffer:
print line
f.close()
### END ###
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of
Joerg Schuster
Sent: 07 March 2005 13:37
To: python-list@python.org
Subject: shuffle the lines of a large file
Hello,
I am looking for a method to "shuffle" the lines of a large file.
I have a corpus of sorted and "uniqed" English sentences that has been
produced with (1):
(1) sort corpus | uniq > corpus.uniq
corpus.uniq is 80G large. The fact that every sentence appears only
once in corpus.uniq plays an important role for the processes
I use to involve my corpus in. Yet, the alphabetical order is an
unwanted side effect of (1): Very often, I do not want (or rather, I
do not have the computational capacities) to apply a program to all of
corpus.uniq. Yet, any series of lines of corpus.uniq is obviously a
very lopsided set of English sentences.
So, it would be very useful to do one of the following things:
- produce corpus.uniq in a such a way that it is not sorted in any way
- shuffle corpus.uniq > corpus.uniq.shuffled
Unfortunately, none of the machines that I may use has 80G RAM.
So, using a dictionary will not help.
Any ideas?
Joerg Schuster
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
On Monday 07 March 2005 14:24, Joerg Schuster wrote:
> Well, I can give you the string, but that will not help:
>
> transduce abc info_dic comp_dic input_file output_file
Several variables like PATH "normally" get reset even when running a non-login
subshell to the standard values from /etc/profile (on Gentoo /etc/env.d/*),
so I guess that you're just having a problem finding the executable for
transduce if that program isn't installed in a path which is always on $PATH.
At least I know this behaviour from some older versions of SuSE; Gentoo with
bash 3.0-r8 and baselayout 1.11.9-r1 does as I would presume and doesn't
reset it (there goes my example). What you might try:
export PATH="/does/not/exist:$PATH"
echo $PATH
--- Path here ---
python
>>> import os
>>> os.system("echo $PATH")
--- Path here, different? ---
and check whether they are any different. You could also do this for other
important variables, such as LD_LIBRARY_PATH. But, all of this is stabbing in
the dark, maybe you can just send the actual error message along next time.
HTH!
--
--- Heiko.
pgpBcN9CKzqop.pgp
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
On Monday 07 March 2005 14:36, Joerg Schuster wrote:
> Any ideas?
The following program should do the trick (filenames are hardcoded, look at
top of file):
### shuffle.py
import random
import shelve
# Open external files needed for data storage.
lines = open("test.dat","r")
lineindex = shelve.open("test.idx")
newlines = open("test.new.dat","w")
# Create an index of all lines of the file in an external flat file DB.
# This means that nothing actually remains in memory, but in an extremely
# efficient (g)dbm flatfile DB.
def makeIdx():
i = 0L
lastpos = 0L
curpos = None
while lines.readline():
# This is after the (\r)\n, which will be stripped() and rewritten
# by writeNewLines().
curpos = long(lines.tell())
lineindex[hex(i)[2:-1]] = "%s:%s" % (hex(lastpos)[2:-1],
hex(curpos-lastpos)[2:-1])
lastpos = curpos
i += 1
return i
maxidx = makeIdx()
# To shuffle the file, just shuffle the index. Problem being: there is no
# random number generator which even remotely has the possibility of yielding
# all possible permutations. Thus, for simplicity: just exchange every element
# in order 1..end with a random element from the rest of the file. This is
# certainly no perfect shuffle, and in case the shuffling is too bad, just
# rerun shuffleIdx() a couple of times.
def shuffleIdx():
oldi = 0L
# Use a while loop, as xrange doesn't work with longs.
while oldi < maxidx:
oi = hex(oldi)[2:-1]
while True:
ni = hex(long(random.randrange(maxidx)))[2:-1]
if ni <> oi:
break
lineindex[oi], lineindex[ni] = lineindex[ni], lineindex[oi]
oldi += 1
shuffleIdx()
# Write out the shuffled file. Do this by just walking the index 0..end.
def writeNewLines():
i = 0L
# Use a while loop, as xrange doesn't work with longs.
while i < maxidx:
# Extract line index and line length from the index file.
lidx, llen = [long(x,16) for x in lineindex[hex(i)[2:-1]].split(":")]
lines.seek(lidx)
line = lines.read(llen).strip()
newlines.write(line+"\n")
i += 1
writeNewLines()
### End shuffle.py
I don't know how fast this program will run, but at least, it does as
told... ;)
--
--- Heiko.
pgpeVbq0wufOV.pgp
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
"Joerg Schuster" <[EMAIL PROTECTED]> writes: >Hello, >I am looking for a method to "shuffle" the lines of a large file. >I have a corpus of sorted and "uniqed" English sentences that has been >produced with (1): >(1) sort corpus | uniq > corpus.uniq >corpus.uniq is 80G large. The fact that every sentence appears only >once in corpus.uniq plays an important role for the processes >I use to involve my corpus in. Yet, the alphabetical order is an >unwanted side effect of (1): Very often, I do not want (or rather, I >do not have the computational capacities) to apply a program to all of >corpus.uniq. Yet, any series of lines of corpus.uniq is obviously a >very lopsided set of English sentences. >So, it would be very useful to do one of the following things: >- produce corpus.uniq in a such a way that it is not sorted in any way >- shuffle corpus.uniq > corpus.uniq.shuffled >Unfortunately, none of the machines that I may use has 80G RAM. >So, using a dictionary will not help. >Any ideas? Instead of shuffling the file itself maybe you could index it (with dbm for instance) and select random lines by using random indexes whenever you need a sample. Eddie -- http://mail.python.org/mailman/listinfo/python-list
RE: shuffle the lines of a large file
Woops typo.
else:
buffer.shuffle()
for line in buffer:
print line
should be
else:
random.shuffle(buffer)
for line in buffer:
print line
of course
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of Alex
Stapleton
Sent: 07 March 2005 14:17
To: Joerg Schuster; python-list@python.org
Subject: RE: shuffle the lines of a large file
Not tested this, run it (or some derivation thereof) over the output to get
increasing randomness.
You will want to keep max_buffered_lines as high as possible really I
imagine. If shuffle() is too intensize
you could itterate over the buffer several times randomly removing and
printing lines until the buffer is empty/suitibly small removing some more
processing overhead.
### START ###
import random
f = open('corpus.uniq')
buffer = []
max_buffered_lines = 1000
for line in f:
if len(buffer) < max_buffered_lines:
buffer.append(line)
else:
buffer.shuffle()
for line in buffer:
print line
random.shuffle(buffer)
for line in buffer:
print line
f.close()
### END ###
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of
Joerg Schuster
Sent: 07 March 2005 13:37
To: python-list@python.org
Subject: shuffle the lines of a large file
Hello,
I am looking for a method to "shuffle" the lines of a large file.
I have a corpus of sorted and "uniqed" English sentences that has been
produced with (1):
(1) sort corpus | uniq > corpus.uniq
corpus.uniq is 80G large. The fact that every sentence appears only
once in corpus.uniq plays an important role for the processes
I use to involve my corpus in. Yet, the alphabetical order is an
unwanted side effect of (1): Very often, I do not want (or rather, I
do not have the computational capacities) to apply a program to all of
corpus.uniq. Yet, any series of lines of corpus.uniq is obviously a
very lopsided set of English sentences.
So, it would be very useful to do one of the following things:
- produce corpus.uniq in a such a way that it is not sorted in any way
- shuffle corpus.uniq > corpus.uniq.shuffled
Unfortunately, none of the machines that I may use has 80G RAM.
So, using a dictionary will not help.
Any ideas?
Joerg Schuster
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: reversed heapification?
Jeff Epler wrote: Can you use something like (untested) class ComparisonReverser: def __init__(self, s): self.s = s def __cmp__(self, o): return cmp(o, self.s) def __lt__... # or whichever operation hashes use then use (ComparisonReverser(f(x)), i, x) as the decorated item instead of (f(x), i, x) Thanks! That *was* untested ;). To avoid infinite recusion (and other bizarre errors), you'd have to compare "o.s" and "self.s". Heapq uses "<=" internally for all comparisons, so it's enough to implement the __le__ method: def __le__(self, other): return other.s <= self.s I actually looked at the C-implementation of heapq in 2.4 and saw that it even provides distinct implementations for min-heaps and max-heaps. It would be so convinient if both of them became part of the module... Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
Replying to oneself is bad, but although the program works, I never intended
to use a shelve to store the data. Better to use anydbm.
So, just replace:
import shelve
by
import anydbm
and
lineindex = shelve.open("test.idx")
by
lineindex = anydbm.open("test.idx","c")
Keep the rest as is.
--
--- Heiko.
pgpbEE81rHUyx.pgp
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
mod subprocess, if you're on 2.4: http://docs.python.org/whatsnew/node8.html -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
"Joerg Schuster" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > I am looking for a method to "shuffle" the lines of a large file. Of the top of my head: decorate, randomize, undecorate. Prepend a suitable large random number or hash to each line and then use sort. You could prepend new line numbers instead but even storing the randomised indexes might use too much memory. -- http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
Joerg Schuster <[EMAIL PROTECTED]> wrote:
> os.system(command)
>
> only works for some values of 'command' on my system (Linux). A certain
> shell command (that *does* run on the command line) does not work when
> called with os.system(). Does anyone know a simple and stable way to
> have *any* string executed by the shell?
The command is exectued through the shell, eg
>>> os.system("sleep 60 > z")
$ ps axf
5121 ?S 0:00 rxvt
5123 pts/77 Ss 0:00 \_ bash
5126 pts/77 S+ 0:00 \_ python
5149 pts/77 S+ 0:00 \_ sh -c sleep 60 > z
5150 pts/77 S+ 0:00 \_ sleep 60
Things to check
1) quoting, python vs shell
2) PATH - check PATH is set the same in shell / python
3) check the whole of the environment
Also if you are using 2.4 check the subprocess module
--
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
--
http://mail.python.org/mailman/listinfo/python-list
Re: Appeal for python developers
Boogieman wrote: > Please include "goto" command in future python realeses > I know that proffesional programers doesn't like to use it, > but for me as newbie it's too hard to get used replacing it > with "while", "def" or other commands I'm assuming you mean "python releases", "professional programmer" and "command" == "keyword"/"statement" I'm sorry for slapping the T-word on you in your last thread. Considering your background I don't think you necessarily want Python. Maybe this is what your looking for: freeBasic : http://sourceforge.net/projects/fbc/ libertyBasic: http://lbpp.sourceforge.net/ I tested the first one and it took me back 15 years in a flash. Cheers, Lars -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible to import a module whose name is contained in a variable?
Gerrit Holl wrote:
Ed Leafe wrote:
On Mar 7, 2005, at 5:23 AM, Michael Hoffman wrote:
Avoiding exec (which is a statement, not a function) is much more
important. Since it executes arbitrary code, you can get unpredictable
results from it.
Is there any way to use __import__ to replace the following:
exec("from %s import *" % modulename)
> No.
Between the following two recipes, it seems unlikely that a simple
"No" is really correct:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/223972
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307772
A variation on some of the code therein should certainly be
able to do the equivalent of "from xx import *". (The addition
of support for __all__ is at least required for full emulation.)
-Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
As far as I can tell, what you ultimately want is to be able to extract
a random ("representative?") subset of sentences. Given the huge size
of data, I would suggest not randomizing the file, but randomizing
accesses to the file. E.g. (sorry for off-the-cuff pseudo python):
[adjust 8196 == 2**13 to your disk block size]
. while True:
. byteno = random.randint(0,length_of_file)
. #align to disk block to avoid unnecessary IO
. byteno = (byteno >> 13) << 13 #zero out the bottom 13 bits
. f.seek(byteno) #set the file pointer to a random position
. bytes = r.read(8196) #read one block
. sentences = bytes.splitlines()[2:-1] #omit ends with partial
lines
. do_something(sentences)
If you only need 1000 sentences, use only one sentence from each block,
if you need 1M, then use them all.
[I hope I understood you problem]
-- george
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible to import a module whose name is contained in a variable?
Ed Leafe wrote:
Is there any way to use __import__ to replace the following:
exec("from %s import *" % modulename)
I shouldn't do this but:
module = __import__(modulename)
try:
for key in module.__all__:
globals()[key] = module[key]
except AttributeError:
globals().update(module.__dict__)
But really, you shouldn't be using from x import * as it
has unpredictable results as well.
http://www.python.org/doc/faq/programming.html#what-are-the-best-practices-for-using-import-in-a-module
--
Michael Hoffman
--
http://mail.python.org/mailman/listinfo/python-list
Re: os.system()
[Nick] > $ ps axf > 5121 ?S 0:00 rxvt > 5123 pts/77 Ss 0:00 \_ bash > 5126 pts/77 S+ 0:00 \_ python > 5149 pts/77 S+ 0:00 \_ sh -c sleep 60 > z > 5150 pts/77 S+ 0:00 \_ sleep 60 Wow, good feature of ps - thanks for the education! I learn something valuable from comp.lang.python every week, and most of it has nothing to do with Python. 8-) -- Richie Hindle [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
Joerg Schuster wrote: Unfortunately, none of the machines that I may use has 80G RAM. So, using a dictionary will not help. Any ideas? Why don't you index the file? I would store the byte-offsets of the beginning of each line into an index file. Then you can generate a random number from 1 to Whatever, go get that index from the index file, then open your text file, seek to that position in the file, read one line, and close the file. Using this process you can then extract a somewhat random set of lines from your 'corpus' text file. You probably should consider making a database of the file, keep the raw text file for sure, but create a converted copy in bsddb or pytables format. Warren -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file - filelist.py (0/1)
On 7 Mar 2005 05:36:32 -0800, rumours say that "Joerg Schuster"
<[EMAIL PROTECTED]> might have written:
>Hello,
>
>I am looking for a method to "shuffle" the lines of a large file.
[snip]
>So, it would be very useful to do one of the following things:
>
>- produce corpus.uniq in a such a way that it is not sorted in any way
>- shuffle corpus.uniq > corpus.uniq.shuffled
>
>Unfortunately, none of the machines that I may use has 80G RAM.
>So, using a dictionary will not help.
To implement your 'shuffle' command in Python, you can do the following
algorithm, with a couple of assumptions:
ASSUMPTION
--
The total line count in your big file is less than sys.maxint.
The algorithm as given works for systems where eol is a single '\n'.
ALGORITHM
-
Create a temporary filelist.FileList fl (see attached file) of
struct.calcsize("q") bytes each (struct.pack and the 'q' format string is your
friend), to hold the offset of each line start in big_file. fl[0] would be 0,
fl[1] would be the length of the first line including its '\n' and so on.
Read once the big_file appending to fl the offset each time (if you need help
with this, let me know).
random.shuffle(fl) # this is tested with the filelist.FileList as given
for offset_as_str in fl:
offset= struct.unpack("q", offset_as_str)[0]
big_file.seek(offset)
sys.stdout.write(big_file.readline())
That's it. Redirect output to your preferred file. No promises for speed
though :)
--
TZOTZIOY, I speak England very best.
"Be strict when sending and tolerant when receiving." (from RFC1958)
I really should keep that in mind when talking with people, actually...
--
http://mail.python.org/mailman/listinfo/python-list
MDaemon Warning - virus found: Returned mail: see transcript for details
*** WARNING ** Este mensaje ha sido analizado por MDaemon AntiVirus y ha encontrado un fichero anexo(s) infectado(s). Por favor revise el reporte de abajo. AttachmentVirus name Action taken -- transcript.zipEmail-Worm.Win32.Mydoom.m Removed ** The message was not delivered due to the following reason: Your message could not be delivered because the destination server was not reachable within the allowed queue period. The amount of time a message is queued before it is returned depends on local configura- tion parameters. Most likely there is a network problem that prevented delivery, but it is also possible that the computer is turned off, or does not have a mail system running right now. Your message was not delivered within 1 days: Mail server 207.218.118.207 is not responding. The following recipients did not receive this message: Please reply to [EMAIL PROTECTED] if you feel this message to be in error. -- http://mail.python.org/mailman/listinfo/python-list
Re: Can a method in one class change an object in another class?
thanks guys. Three good answers, each slightly different, but giving me good food for thought. Obviously my example was a trivial one, but I wanted to isolate the behaviour I'm seeing in my real app. I now have some good ideas for moving forward! cheers S -- http://mail.python.org/mailman/listinfo/python-list
Re: parsing a date string
Am Sun, 06 Mar 2005 19:35:23 + schrieb MikeyG:
> Hi,
>
> I have a date string in the ctime() format ('Sat Mar 5 10:38:07 2005')
> and I want to know how long ago that was in whole days.
>
> So far I have:
>
> import time
> import datetime
> age =
> (datetime.date.fromtimestamp(time.mktime(time.strptime(date.strip( -
> datetime.date.today()).days
>
> Which is an absurd number of function calls and is possibly not even
> correct (something in the docs about mktime() taking localtime whereas
> my string is in UTC)
Try %Z:
time.mktime(time.strptime("Sat Mar 5 10:38:07 2005 UTC",
"%a %b %d %H:%M:%S %Y %Z"))
--> 1110015487.0
time.mktime(time.strptime("Sat Mar 5 10:38:07 2005 CEST",
"%a %b %d %H:%M:%S %Y %Z"))
--> 1110011887.0
HTH,
Thomas
--
Thomas Güttler, http://www.thomas-guettler.de/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python & Tkinter ?
Pete wrote: > Never seen this YUM thing but it's pretty trick. more here: http://linux.duke.edu/projects/yum/ -- http://mail.python.org/mailman/listinfo/python-list
Re: intigrate the PyGame module with my Python
1. Downloaded the windows binary for python 1.5.2 from python.org. Pygame uses Python 1.5.2 still!? :-) Oi. Warren -- http://mail.python.org/mailman/listinfo/python-list
Re: Geneator/Iterator Nesting Problem - Any Ideas? 2.4
For those that are interested I ran a performance comparison of various
functions for wrapping sql results in a interables and generators. The
results are below and the code is at the very bottom.
Surprisinly (in a happy way) the functional version of the batch
retrieve comes in a very close second beating out two other common
iterative approaches. However the winner by a small margin is still an
iterative routine. The elegance of the more functional approach (thanks
to Steve) and its excellent performance makes it a clear winner in my
mind.
The other intersting conclusion is that batch fetching results gives a
fairly significant and real performance boost. Its not just academic.
The Winner:
def resultset_functional_batchgenerator(cursor, size=100):
for results in iter(lambda: cursor.fetchmany(size), []):
for rec in results:
yield rec
Test Results (P2.4 IBM T41 Thinkpad):
Ordered by: cumulative time
List reduced from 57 to 7 due to restriction <'test_'>
ncalls tottime percall cumtime percall filename:lineno(function)
12.1402.140 54.002 54.002
PerformanceTestGenerators.py:102(test_resultset_functional_generator)
11.9571.957 45.484 45.484
PerformanceTestGenerators.py:98(test_resultset_iterative_generator)
12.4332.433 41.844 41.844
PerformanceTestGenerators.py:94(test_resultset_iterator)
11.9301.930 39.793 39.793
PerformanceTestGenerators.py:110(test_resultset_iterative_batchgenerator_2)
11.7341.734 35.561 35.561
PerformanceTestGenerators.py:114(test_resultset_iterative_batchgenerator_3)
11.9801.980 34.579 34.579
PerformanceTestGenerators.py:118(test_resultset_functional_batchgenerator)
11.7801.780 31.696 31.696
PerformanceTestGenerators.py:106(test_resultset_iterative_batchgenerator_1)
Code:
import unittest
import odbc
import profile
import pstats
class resultset_iterator:
"Iterate over the recordset and frees the cursor afterwards."
def __init__(self, cursor):
self.cursor = cursor
def __iter__(self): return self
def next(self):
rec = self.cursor.fetchone()
if not rec:
raise StopIteration
return rec
def resultset_iterative_generator(cursor):
rec = cursor.fetchone();
while rec:
yield rec
rec = cursor.fetchone();
def resultset_functional_generator(cursor):
for rec in iter(lambda: cursor.fetchone(), None):
yield rec
def resultset_iterative_batchgenerator_1(cursor, size=100):
results = cursor.fetchmany(size)
while results:
for rec in results:
yield rec
results = cursor.fetchmany(size)
def resultset_iterative_batchgenerator_2(cursor, arraysize=100):
'An iterator that uses fetchmany to keep memory usage down'
done = False
while not done:
results = cursor.fetchmany(arraysize)
if results == []:
done = True
for result in results:
yield result
def resultset_iterative_batchgenerator_3(cursor, size=100):
while True:
results = cursor.fetchmany(size)
if not results:
break
for rec in results:
yield rec
def resultset_functional_batchgenerator(cursor, size=100):
for results in iter(lambda: cursor.fetchmany(size), []):
for rec in results:
yield rec
class testResultSetGenerators(unittest.TestCase):
connectstring = "*REMOVED*"
sql = "*REMOVED*"
def setUp(self):
self.con = odbc.odbc(self.connectstring)
self.cur = self.con.cursor()
self.cur.execute(self.sql)
def tearDown(self):
self.cur.close()
self.con.close()
def test_resultset_iterator(self):
for row in resultset_iterator(self.cur):
pass
def test_resultset_iterative_generator(self):
for row in resultset_iterative_generator(self.cur):
pass
def test_resultset_functional_generator(self):
for row in resultset_functional_generator(self.cur):
pass
def test_resultset_iterative_batchgenerator_1(self):
for row in resultset_iterative_batchgenerator_1(self.cur):
pass
def test_resultset_iterative_batchgenerator_2(self):
for row in resultset_iterative_batchgenerator_2(self.cur):
pass
def test_resultset_iterative_batchgenerator_3(self):
for row in resultset_iterative_batchgenerator_3(self.cur):
pass
def test_resultset_functional_batchgenerator(self):
for row in resultset_functional_batchgenerator(self.cur):
pass
if __name__ == '__main__':
suite = unittest.makeSuite(testResultSetGenerators)
profile.run('unittest.TextTestRunner(verbosity=2).run(suite)',
'c:\\temp\\profile')
p = pstats.Stats('c:\\temp\\profile')
p.strip_dirs().sort_stats('cumulative').print_stats('test_')
--
htt
Re: intigrate the PyGame module with my Python
Warren Postma wrote: 1. Downloaded the windows binary for python 1.5.2 from python.org. Pygame uses Python 1.5.2 still!? :-) Oi. Nah, must have been a typo, as www.pygame.org lists Windows installers for Python 2.2, 2.3, and 2.4: http://www.pygame.org/download.shtml -Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4 removes None data type?
Michael Hoffman wrote: The fact that True and False are not constants? Yowza. a = True b = False False = a True = b if (1==2)==True: print "Doom" -- http://mail.python.org/mailman/listinfo/python-list
Re: bsddb for k, v in db.items(): do order the numbers ?
uhm i'm trying to make a very simple but large database: Let's say I want these fields : |name|age|country| Then I can't do this because I use the same key db["name"] = 'piet' db["age"] = '20' db["country"] = 'nl' #same keys so it wil overwrite db["name"] = 'jan' db["age"] = '40' db["country"] = 'eng' But how does other people use bsddb then ? - with a hidden |int like below ? db["name|0"] = 'jan' db["age|1"] = '40' db["country|2"] = 'eng' - do a little math to first is name sec is age third is country db["0"] = 'jan' db["1"] = '40' db["2"] = 'eng' pointer=0 for k, v in db.items(): if pointer =3: poiner = 0 #next 3 fields -- I like bsddb because of the speed and it can handle big files, but what is the normal way of using it ? Thanks for helping -- http://mail.python.org/mailman/listinfo/python-list
Re: Geneator/Iterator Nesting Problem - Any Ideas? 2.4
ChaosKCW wrote: > def resultset_functional_generator(cursor): > for rec in iter(lambda: cursor.fetchone(), None): > yield rec This can be simplified to def resultset_functional_generator(cursor): return iter(cursor.fetchone, None) It should be a bit faster, too. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4 removes None data type?
Michael Hoffman wrote: The fact that True and False are not constants? Yowza. a = True b = False False = a True = b if (1==2)==True: print "Doom" -- http://mail.python.org/mailman/listinfo/python-list
Re: Ruby on Rails or Perl's Maypole..is there a Python equivalent
John J. Lee wrote: I know mono runs on linux but I want nothing to do with it unless absolutely necessary. > Gary Nutbeam <[EMAIL PROTECTED]> writes: >> D H wrote: > [...] >> > Check out Castle on Rails for .NET/Mono. It is still in early >> > development, but you can use it with C#, VB, or boo, and I'm sure >> > eventually with IronPython as well. >> >> Thanks for the feedback. I should have been more specific though and >> mentioned this has done on Linux (client and server). > > Mono runs on Linux. > > > John -- http://mail.python.org/mailman/listinfo/python-list
Re: Ruby on Rails or Perl's Maypole..is there a Python equivalent
Learning Ruby to use Rails is tempting. Iwan van der Kleyn wrote: > Gary Nutbeam wrote: >> needing to learn Ruby. > > But why wouldn't you just use Rails and learn Ruby in the process? The > "effort" required to learn Ruby pales in comparisson to the advantages > using Ruby on Rails might give you, imho. > > Ruby is an excellent language, not much different from Python with its > own set of advantages and problems (I really mis python's white-space > indentation for example, but that is fully compensated by Ruby's nice > OOP features). With a book like "Programming Ruby" you would be up to > speed in a few days. > > Rails gives you much more than a comparable set of Python libraries > which are gobled together with sticky tape. It provides you not just > with an superbly integrated and consistent set of components. Rails > gives you: > - (real) automation (take a look at scaffolding for quick prototyping) > - terrific documentation (the videos are *not* a gimmick, for example) > - an enthousiastic, supportive user community (that alone is an > incredible help and time saver) > > have fun, > > Iwan -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
Thanks to all. This thread shows again that Python's best feature is comp.lang.python. Jörg -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4 removes None data type?
Warren Postma wrote: Michael Hoffman wrote: The fact that True and False are not constants? Yowza. a = True b = False False = a True = b if (1==2)==True: print "Doom" Why stop there when you can really cause some doom: py> import __builtin__ as bltin py> bltin.True, bltin.False = bltin.False, bltin.True As an example of what doom this causes, try typing it at the interactive prompt, and then see if you can enter anything else. I can't -- the prompt just locks up. Woo-hoo! Another way to shoot myself (and my users) in the foot! =) STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
Joerg Schuster wrote: Thanks to all. This thread shows again that Python's best feature is comp.lang.python. +1 QOTW STeVe -- http://mail.python.org/mailman/listinfo/python-list
RE: shuffle the lines of a large file
Title: RE: shuffle the lines of a large file [Joerg Schuster] #- Thanks to all. This thread shows again that Python's best feature is #- comp.lang.python. QOTW! QOTW! . Facundo Bitácora De Vuelo: http://www.taniquetil.com.ar/plog PyAr - Python Argentina: http://pyar.decode.com.ar/ -- http://mail.python.org/mailman/listinfo/python-list
Re: bsddb for k, v in db.items(): do order the numbers ?
[EMAIL PROTECTED] wrote:
uhm i'm trying to make a very simple but large database:
Let's say I want these fields : |name|age|country|
Then I can't do this because I use the same key
db["name"] = 'piet'
db["age"] = '20'
db["country"] = 'nl'
#same keys so it wil overwrite
db["name"] = 'jan'
db["age"] = '40'
db["country"] = 'eng'
But how does other people use bsddb then ?
- with a hidden |int like below ?
db["name|0"] = 'jan'
db["age|1"] = '40'
db["country|2"] = 'eng'
- do a little math to
first is name
sec is age
third is country
db["0"] = 'jan'
db["1"] = '40'
db["2"] = 'eng'
pointer=0
for k, v in db.items():
if pointer =3:
poiner = 0
#next 3 fields
--
I like bsddb because of the speed and it can handle big files,
but what is the normal way of using it ?
I don't know about normal but I'd probably do something like
db['piet'] = repr(['piet', 20, 'nl'])
or maybe
db['jan'] = repr({"name":'jan', "age":40, "country":'eng'})
That should hold until Piet #2 comes along, then I might add another
level of lists...
With more complicated data I'd do as the docs say and take a look at
marshal or pickle instead of using repr(). And use a class instead of
lists or dicts...
/ug
--
http://mail.python.org/mailman/listinfo/python-list
Catching all methods before they execute
I have run into some cases where I would like to run a class method anytime any class method is invoked. That is, if I write x.foo then it will be the same as writing x.bar x.foo for any method in class x (with the possible exception of 'bar'). The first few times I wanted to print out a data structure for debugging purposes. Most recently it was to save a data structure to allow "undo" operations. I've realized that this is a generalized problem and I was hoping that someone could point me to the correct hook to accomplish this (python seems to have a hook for everything). -- http://mail.python.org/mailman/listinfo/python-list
Re: Catching all methods before they execute
[EMAIL PROTECTED] wrote: I have run into some cases where I would like to run a class method anytime any class method is invoked. Perhaps you want __getattribute__ on a new-style class? -- Michael Hoffman -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4 removes None data type?
Steven Bethard said unto the world upon 2005-03-07 11:55: Warren Postma wrote: Michael Hoffman wrote: The fact that True and False are not constants? Yowza. a = True b = False False = a True = b if (1==2)==True: print "Doom" Why stop there when you can really cause some doom: py> import __builtin__ as bltin py> bltin.True, bltin.False = bltin.False, bltin.True As an example of what doom this causes, try typing it at the interactive prompt, and then see if you can enter anything else. I can't -- the prompt just locks up. Woo-hoo! Another way to shoot myself (and my users) in the foot! =) STeVe Hi all, just tried this in IDLE: IDLE 1.1 >>> import __builtin__ as bltin >>> bltin.True, bltin.False = bltin.False, bltin.True >>> RESTART The restart was done by IDLE itself -- poor thing just gave up. So, it looks like Steve has found a `commit suicide' command for an IDLE shell :-) Best to all, Brian vdB -- http://mail.python.org/mailman/listinfo/python-list
mod_python & "please recompile it with -DEAPI" apache warning
In regards to mod_python, I keep getting this warning when I do apachectl configtest: [warn] Loaded DSO libexec/mod_python.so uses plain Apache 1.3 API, this module might crash under EAPI! (please recompile it with -DEAPI) So I googled what I can, and have read various responses. Most say you just need to recompile the module with the DEAPI flag. I was getting the same message for mod_php, and fixed that. However, with the same config command, it doesn't fix it for mod_python: 1) CFLAGS="-DEAPI -fPIC" ./configure --with-apxs=/www/bin/apxs make make install I later read someone trying this to the the MakeFile after configuring: 2) added to MakeFile after config: EXTRA_CFLAGS=-DEAPI There was even someone claiming this is a documented bug - implying not to worry about it? Any help would be much appreciated. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
global variables
hi; i have a project with multiple files in it. I need to have a varaible that will contain a value that will be modified in one file, and when coming back to the same file it should retain the same value. The way am doing it right now, python is complaining about the variable being non-global any clue? thanks m.smadi -- http://mail.python.org/mailman/listinfo/python-list
Re: Catching all methods before they execute
>[EMAIL PROTECTED] wrote: >> I have run into some cases where I would like to run a class method >> anytime any class method is invoked. > >Perhaps you want __getattribute__ on a new-style class? >-- >Michael Hoffman Perhaps I do. The docs say that __getattribute__ is called on all attribute references, so I tried to make an undoable list as follows: % cat Undoable.py class Undoable(object): def __init__(self, superclass): self.superclass = superclass print "__init__" def __getattribute__(self, name): print "__getattribute__" self.SaveState(self) self.superclass.__getattribute__(self, name) def SaveState(self): print "SaveState" def RestoreState(self): pass l = Undoable(list) l = [1, 2, 3] print l.count(1) % python Undoable.py __init__ 1 It appears that __init__ in Undoable is called, count() in list is called, but not __getattribute__ or SaveState in Undoable. What don't I understand? -- http://mail.python.org/mailman/listinfo/python-list
zipimport IOError reading Modules.zip in Mac standalone app
We have some users of our application getting error messages like: IOError: zipimport: can not open file /Volumes/MyApp/MyApp.app/Contents/Resources/Modules.zip This only happens on our Mac version - the Windows version seems fine. Our build is still using bundlebuilder, which creates Modules.zip. I want to upgrade to py2app, but have not yet done so (we are using py2exe for the Windows version). Only a few (7) of our users have had these IOError problems other users are running fine. The errors they were all either: in shelve.__init__ in encodings/__init__.py or in the import to one of our own modules. When I look into the Modules.zip file, I see shelve.pyo encodings/__init__.pyo and our own module's pyo file So, I don't understand why we get these zipimport IOErrors. The application launches more than one process that uses the embedded python (we started using the new subprocess module in this release). Could that be a problem? Could it be that more than one process is trying to read Modules.zip at the same time? If so, what do we do - I don't want to have to include a separate version of the code for each subprocess. Any other suggestion? I need this fixed ASAP. Many thanks, Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: survey
I'd like to know how scripts are processed, whether they are compiled into bytecode, how memory is allocated for variables, how scripting languages perform compared to each other, etc. --- Cameron Laird <[EMAIL PROTECTED]> wrote: > In article <[EMAIL PROTECTED]>, > Peter Hansen <[EMAIL PROTECTED]> wrote: > >Dave Zhu wrote: > >> Hello All, > >> > >> Is there any survey on scripting languages? I > would > >> like to get information on several scripting > languages > >> including Python, Perl, Ruby, Tcl, etc. > > > >What kind of information? ... > > Precisely. There are whole books http://www.amazon.com/exec/obidos/ASIN/1578700108/002-9216984-6160028 > > > on this subject; there are breezy sketches http://www.itworld.com/AppDev/4061/swol-0202-regex/ > > > you can read in a single setting; and 'most > everything > between. Who wants to know? Is your real interest > to > know what the job market for each language is, or > the > syntax of arithmetic in each language, or which one > has > the best Conferences, or ...? > > http://www.catb.org/~esr/faqs/smart-questions.html > > __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -- http://mail.python.org/mailman/listinfo/python-list
Unicode BOM marks
Hi, For the first time in my programmer life, I have to take care of character encoding. I have a question about the BOM marks. If I understand well, into the UTF-8 unicode binary representation, some systems add at the beginning of the file a BOM mark (Windows?), some don't. (Linux?). Therefore, the exact same text encoded in the same UTF-8 will result in two different binary files, and of a slightly different length. Right ? I guess that this leading BOM mark are special marking bytes that can't be, in no way, decoded as valid text. Right ? (I really really hope the answer is yes otherwise we're in hell when moving file from one platform to another, even with the same Unicode encoding). I also guess that this leading BOM mark is silently ignored by any unicode aware file stream reader to which we already indicated that the file follows the UTF-8 encoding standard. Right ? If so, is it the case with the python codecs decoder ? In python documentation, I see theseconstants. The documentation is not clear to which encoding these constants apply. Here's my understanding : BOM : UTF-8 only or UTF-8 and UTF-32 ? BOM_BE : UTF-8 only or UTF-8 and UTF-32 ? BOM_LE : UTF-8 only or UTF-8 and UTF-32 ? BOM_UTF8 : UTF-8 only BOM_UTF16 : UTF-16 only BOM_UTF16_BE : UTF-16 only BOM_UTF16_LE : UTF-16 only BOM_UTF32 : UTF-32 only BOM_UTF32_BE : UTF-32 only BOM_UTF32_LE : UTF-32 only Why should I need these constants if codecs decoder can handle them without my help, only specifying the encoding ? Thank you Francis Girard Python tells me to use an encoding declaration at the top of my files (the message is referring to http://www.python.org/peps/pep-0263.html). I expected to see there a list of acceptable -- http://mail.python.org/mailman/listinfo/python-list
hidden attribute on Windows files
How do I enable the hidden attribute when creating files on Windows computers? I'd *really* prefer to do from the standard Python installer (no win32 extensions). Any tips? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Catching all methods before they execute
[EMAIL PROTECTED] wrote: l = Undoable(list) l = [1, 2, 3] You just rebound l, so it no longer refers to an Undoable, it refers to a list. This design won't work, you need something more like: l = Undoable([1, 2, 3]) There were a few other pitfalls in your design... Here, try something like this instead: class SurrogateNotInitedError(exceptions.AttributeError): pass class Surrogate(object): """ the data is stored in _data >>> list1 = [0, 1, 2, 3] >>> list2 = [4, 5, 6, 7] >>> surrogate = Surrogate(list1) >>> surrogate.reverse() >>> list1 [3, 2, 1, 0] >>> surrogate._data = list2 >>> surrogate.append(8) >>> list2 [4, 5, 6, 7, 8] """ def __init__(self, data): self._data = data def __getattr__(self, name): if name == "_data": raise SurrogateNotInitedError, name else: try: return getattr(self._data, name) except SurrogateNotInitedError: raise SurrogateNotInitedError, name You can modify this to make an UndoableSurrogate with appropriate saving of state. Note the use of __getattr__ instead of __getattribute__. The latter is not needed since there are not any attributes defined on Surrogate instead. -- Michael Hoffman -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
[Joerg Schuster] > I am looking for a method to "shuffle" the lines of a large file. If speed and space are not a concern, I would be tempted to presume that this can be organised without too much difficulty. However, looking for speed handling a big file, while keeping equiprobability of all possible permutations, might be sensibly more difficult. I vaguely remember having read something along these lines (not shuffling as you mean it, but still, reorganising a lengthy file) in Knuth's "Art of Computer Programming", in one of the exercises within the chapter on Sorting methods (volume 3). That's long ago, but if I remember well, Knuth did not consider this as an easy exercise. -- François Pinard http://pinard.progiciels-bpi.ca -- http://mail.python.org/mailman/listinfo/python-list
Re: shuffle the lines of a large file
[Heiko Wundram] > Replying to oneself is bad, [...] Not necessarily. :-) -- François Pinard http://pinard.progiciels-bpi.ca -- http://mail.python.org/mailman/listinfo/python-list
Re: python -i (interactive environment)
Michael Hoffman wrote: Joe wrote: I want the script to decide whether to fall back to the interactive prompt. You solution makes it ALWAYS fall back to the interactive prompt. Actually, using sys.exit() means the program can exit even if python -i is used. You can use: import code code.interact() which emulates the interactive prompt. Unfortunately it does so in an entirely new namespace, thereby losing the advantage of -i - namely, that you can investigate the program's namespace after it's terminated. regards Steve -- http://mail.python.org/mailman/listinfo/python-list
Dr. Dobb's Python-URL! - weekly Python news and links (Mar 7)
QOTW: "Really, of course, the only things you need to make explicit are the ones that readers don't understand." -- Steve Holden "Working with unicode objects in Python is so transparent, it's easy to forget about what a C extension would likely want." -- Kevin Dangoor "You take leadership in a small area, and you keep it as long as you care to." -- Max M, on the slender rewards of open-source authorship Even with everything else going on at PyCon2005--dynamite speakers, thrilling Sprints, and so on--the most newsworthy event of the week likely will be revelations about IronPython's schedule. When will it be possible to create .NET services in Python? Keep in mind that PythonNet already lets Python applications be .NET clients: http://www.zope.org/Members/Brian/PythonNet/index_html http://python.org/pycon/2005/keynotes.html Brett C. begins his farewell tour from python-dev summation: http://groups-beta.google.com/group/comp.lang.python/msg/99e13efba90ca590 Python has a Security Response Team, python-dev has new summarizers, CPython might soon have a new "throwaway list" optimization, and those only hint at the range of progress python-dev has made in the last fortnight: http://www.python.org/dev/summary/2005-02-01_2005-02-14.html Skip Montanaro and Diez Roggisch write with precision on use of urllib, and particularly its user-agent refinement: http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/535ada773d6f6227/ Python is a good extension language, in the sense that an application or parts of it can be made scriptable under the control of an end-user. It's easy to do so--but be careful to promote "safe plugging": http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/620954f7d9767af6/ Jeff Epler exhibits a definition which automates remote control of Tkinter processes: http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/2ac9a3f8309d183a/ ChaosKCW offers timings and brief commentary on alternative wrappings of SQL access with iterables and generators: http://groups-beta.google.com/group/comp.lang.python/msg/7ff516d7d9387dad Python is flexible. You can create applications, you can work interactively, you can interactively manage applications, you can create applications that offer interaction under programmatic control, you can ...: http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/76826e3be040d63d/ One of the fundamentals of Python's origin was that the language play well with others (an innovation, in that context). Among the many instances of teamwork between Python and other languages that turn up daily is Python's success in development of assemblers: http://groups-beta.google.com/group/comp.lang.python.announce/browse_thread/thread/ce3329a8e408c202/ Everything Python-related you want is probably one or two clicks away in these pages: Python.org's Python Language Website is the traditional center of Pythonia http://www.python.org Notice especially the master FAQ http://www.python.org/doc/FAQ.html PythonWare complements the digest you're reading with the marvelous daily python url http://www.pythonware.com/daily Mygale is a news-gathering webcrawler that specializes in (new) World-Wide Web articles related to Python. http://www.awaretek.com/nowak/mygale.html While cosmetically similar, Mygale and the Daily Python-URL are utterly different in their technologies and generally in their results. For far, FAR more Python reading than any one mind should absorb, much of it quite interesting, several pages index much of the universe of Pybloggers. http://lowlife.jp/cgi-bin/moin.cgi/PythonProgrammersWeblog http://www.planetpython.org/ http://mechanicalcat.net/pyblagg.html comp.lang.python.announce announces new Python software. Be sure to scan this newsgroup weekly. http://groups.google.com/groups?oi=djq&as_ugroup=comp.lang.python.announce Brett Cannon continues the marvelous tradition established by Andrew Kuchling and Michael Hudson of intelligently summarizing action on the python-dev mailing list once every other week. http://www.python.org/dev/summary/ The Python Package Index catalogues packages. http://www.python.org/pypi/ The somewhat older Vaults of Parnassus ambitiously collects references to all sorts of Python resources. http://www.vex.net/~x/parnassus/ Much of Python's real work takes place on Special-Interest Group mailing lists http://www.python.org/sigs/ The Python Business Forum "further[s] the interests of
Python 2.4 / WinXP / distutils error (cookbook example)
Hi, I've installed Python 2.4 on Windows XP and walked through the Alex Martelli ASPN cookbook example at: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66509 This is a recipe for a simple extension type for Python. When I try to build and install it, however, I get an error: c:\temp\el>python setup.py install running install running build running build_ext error: The .NET Framework SDK needs to be installed before building extensions for Python. I have Visual Studio .NET Professional installed. Can anyone point me in the right direction? Thanks! Marcus -- http://mail.python.org/mailman/listinfo/python-list
autoexecution in Windows
How does one make a Python program auto-execute in Windows? Earl -- http://mail.python.org/mailman/listinfo/python-list
Re: function with a state
Reinhold Birkenfeld wrote: > or with a default function argument: > > > class Dummy: pass > > def myFun(globe=Dummy()): > try:globe.globe += 1 > except: globe.globe = 1 > > return globe.globe A quicker way: def myFun(globe=[0]): globe[0] += 1 return globe[0] Reinhold -- http://mail.python.org/mailman/listinfo/python-list
Re: autoexecution in Windows
Le Mon, 07 Mar 2005 13:25:35 -0700, Earl Eiland a écrit : > How does one make a Python program auto-execute in Windows? > > Earl > write a virus ? :-) What do you mean by « auto-execute » ? Regards -- http://mail.python.org/mailman/listinfo/python-list
Re: hidden attribute on Windows files
rbt wrote:
> How do I enable the hidden attribute when creating files on Windows
> computers? I'd *really* prefer to do from the standard Python installer
> (no win32 extensions). Any tips?
You could do
os.system('attrib +h hidethis.txt')
but that only works if hidethis already exists.
--
"Codito ergo sum"
Roel Schroeven
--
http://mail.python.org/mailman/listinfo/python-list
Re: autoexecution in Windows
In Linux, if I make the first line #!/path/to/Python, all I have to do to execute the program is type ./FileName (assuming my pwd is the same as FileName). what's the Windows equivalent? Earl On Mon, 2005-03-07 at 13:36, F. Petitjean wrote: > Le Mon, 07 Mar 2005 13:25:35 -0700, Earl Eiland a Ãcrit : > > How does one make a Python program auto-execute in Windows? > > > > Earl > > > write a virus ? :-) > > What do you mean by  auto-execute  ? > > Regards -- http://mail.python.org/mailman/listinfo/python-list
pyconfig.h
I have installed Python-2.3.3 in the SUSE Linux 9.1 system and am trying to rebuild an application rpm that was implemented in Python, the defines in the pyconfig.h seem to be related to Microsoft 32-bit and 64-bit environment, do I need these defines in the linux environment? what is the purpose of this header file? Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode BOM marks
Francis Girard wrote:
If I understand well, into the UTF-8 unicode binary representation, some
systems add at the beginning of the file a BOM mark (Windows?), some don't.
(Linux?). Therefore, the exact same text encoded in the same UTF-8 will
result in two different binary files, and of a slightly different length.
Right ?
Mostly correct. I would prefer if people referred to the thing not as
"BOM" but as "UTF-8 signature", atleast in the context of UTF-8, as
UTF-8 has no byte-order issues that a "byte order mark" would deal with.
(it is correct to call it "BOM" in the context of UTF-16 or UTF-32).
Also, "some systems" is inadequate. It is not so much the operating
system that decides to add or leave out the UTF-8 signature, but much
more the application writing the file. Any high-quality tool will accept
the file with or without signature, whether it is a tool on Windows
or a tool on Unix.
I personally would write my applications so that they put the signature
into files that cannot be concatenated meaningfully (since the
signature simplifies encoding auto-detection) and leave out the
signature from files which can be concatenated (as concatenating the
files will put the signature in the middle of a file).
I guess that this leading BOM mark are special marking bytes that can't be, in
no way, decoded as valid text.
Right ?
Wrong. The BOM mark decodes as U+FEFF:
>>> codecs.BOM_UTF8.decode("utf-8")
u'\ufeff'
This is what makes it a byte order mark: in UTF-16, you can tell the
byte order by checking whether it is FEFF or FFFE. The character U+FFFE
is an invalid character, which cannot be decoded as valid text
(although the Python codec will decode it as invalid text).
I also guess that this leading BOM mark is silently ignored by any unicode
aware file stream reader to which we already indicated that the file follows
the UTF-8 encoding standard.
Right ?
No. It should eventually be ignored by the application, but whether the
stream reader special-cases it or not is depends on application needs.
If so, is it the case with the python codecs decoder ?
No; the Python UTF-8 codec is unaware of the UTF-8 signature. It reports
it to the application when it finds it, and it will never generate the
signature on its own. So processing the UTF-8 signature is left to the
application in Python.
In python documentation, I see theseconstants. The documentation is not clear
to which encoding these constants apply. Here's my understanding :
BOM : UTF-8 only or UTF-8 and UTF-32 ?
UTF-16.
BOM_BE : UTF-8 only or UTF-8 and UTF-32 ?
BOM_LE : UTF-8 only or UTF-8 and UTF-32 ?
UTF-16
Why should I need these constants if codecs decoder can handle them without my
help, only specifying the encoding ?
Well, because the codecs don't. It might be useful to add a
"utf-8-signature" codec some day, which generates the signature on
encoding, and removes it on decoding.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4 / WinXP / distutils error (cookbook example)
[EMAIL PROTECTED] wrote: I have Visual Studio .NET Professional installed. Can anyone point me in the right direction? There are several solutions, but one is to install Visual Studio .NET 2003 (which is different from Visual Studio .NET, also referred to as VS.NET 2002). Microsoft managed to rename the C library (CRT) between 2002 (msvcr7.dll) and 2003 (msvcr71.dll), and you need to make sure your extension uses the same CRT as the one used to build Python. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: how to execute Python in VIM
thanks Aaron i've changed that, but this time, even worse... when i press F5, the pop-up windows appears, and then, it disppears very quickly...(less than 1 second) i cant see anything~ :( -- http://mail.python.org/mailman/listinfo/python-list
Re: autoexecution in Windows
Earl Eiland wrote: How does one make a Python program auto-execute in Windows? Earl No program (python or other) can just arbitrarily execute. A user has to click it or a cron-like utility (Task Scheduler) has to execute it at a set time. registry entries (such as run) can execute programs too. Also, proper Windows services can be configured to start at boot. Again, nothing can just arbitrarily execute. If it could, viruses would be a *nightmare* -- http://mail.python.org/mailman/listinfo/python-list
Re: hidden attribute on Windows files
rbt wrote: How do I enable the hidden attribute when creating files on Windows computers? I'd *really* prefer to do from the standard Python installer (no win32 extensions). Any tips? With pure Python and just the standard installer, you need to invoke attrib.exe, as attrib.exe +H Use os.system to invoke that command, make sure you get the quoting right in case pathname might contain spaces. attrib.exe is located in the system folder (windows\system32), so it should be on the PATH. HTH, Martin -- http://mail.python.org/mailman/listinfo/python-list
