Re: Break large file down into multiple files

[redbaron]

> New to python I have a large file that I need to break up into
> multiple smaller files. I need to break the large file into sections
> where there are 65535 lines and then write those sections to seperate
> files.

If your lines are variable-length, then look at itertools recipes.

from itertools import izip_longest

def grouper(n, iterable, fillvalue=None):
"grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
args = [iter(iterable)] * n
return izip_longest(fillvalue=fillvalue, *args)

with open("/file","r") as f:
for lines in grouper(65535,f,""):
data_to_write = '\n'.join(lines).rstrip("\n")
...

...

--
http://mail.python.org/mailman/listinfo/python-list

Re: thread. question

[Tim Wintle]

On Mon, 2009-02-09 at 21:02 +0100, Christian Heimes wrote:
> The module was renamed to _thread to stop people from using it directly.
> The extension module is the interface to some low level types and
> functions. Especially the usage of thread.start_new_thread is
> problematic, since it bypasses Python's high level threading API.

Ok, I'll take that in - I'll have a look around the threading module to
see what it is that it bypasses so I can persuade myself that _thread is
bad ...

> For the rest I have to agree with Jean-Paul. If you need performance
> don't use threads! Threads and performance are orthogonal -- sometimes
> they are even contradictorily.

I've taken that in too - "performance" here is over a non-uniform
workload (essentially a web server), but I'm probably best offloading
just about all the work to a secondary process (already passed most of
it) and dealing with requests in a completely synchronous process.

Tim

--
http://mail.python.org/mailman/listinfo/python-list

Re: something wrong with isinstance

[Bruno Desthuilliers]

redbaron a écrit :

Don't really sure, but try to define your class as new-style one.


isinstance() works as well with classic classes.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Invoking CutePDF from within Python

[Tim Golden]

John Henry wrote:
I have a need to invoke CutePDF from within a Python program.  


All I need is to say "Print this to CUTEPDF and store as xyz.pdf".



Private Sub Print_PDF()

 lRetVal = RegCreateKeyEx(HKEY_CURRENT_USER, "Software\Custom PDF
Printer",  _
0&, vbNullString, REG_OPTION_NON_VOLATILE,
KEY_ALL_ACCESS, _
0&, hKey, lRetVal)
 sValue = "C:\Sample.pdf"
 RegSetValueExString hKey, "OutputFile", 0&, REG_SZ, sValue, Len
(sValue)
 sValue = "1"
 RegSetValueExString hKey, "BypassSaveAs", 0&, REG_SZ, sValue, Len
(sValue)

 Dim worddoc As Word.Document
 Set worddoc = wordapp.Documents.Open("C:\Sample.doc")
 wordapp.ActivePrinter = "Custom PDF Printer"
 wordapp.PrintOut
 worddoc.Close

 sValue = "0"
 RegSetValueExString hKey, "BypassSaveAs", 0&, REG_SZ, sValue, Len
(sValue)
 RegCloseKey (hKey)



Direct translation (untested since I don't have CutePDF installed)


import win32api
import win32con
import win32com

hKey, ret = win32api.RegCreateKeyEx (
 win32con.HKEY_CURRENT_USER, 
 "Software\Custom PDF Printer",

 win32con.KEY_ALL_ACCESS
)
win32api.RegSetValueEx (hKey, "OutputFile", None, win32con.REG_SZ, 
r"c:\sample.pdf")
win32api.RegSetValueEx (hKey, "BypassSaveAs", None, win32con.REG_SZ, r"1")

word = win32com.client.gencache.EnsureDispatch ("Word.Application")
doc = word.Documents.Open (r"c:\sample.doc")
doc.ActivePrinter = "Custom PDF Printer"
word.Printout ()
doc.Close ()

win32api.RegSetValueEx (hKey, "BypassSaveAs", None, win32con.REG_SZ, r"0")




FWIW, I usually generate PDFs by printing to a Postscript printer
(some random Apple Laserthingy) and then using Ghostscript to
do the conversion. I'm happy to post if you're interested.

TJG


--
http://mail.python.org/mailman/listinfo/python-list

Re: something wrong with isinstance

[Bruno Desthuilliers]

maksym.ka...@gmail.com a écrit :

Hi there.
now i'm a complete newbie for python, and maybe my problem is stupid
but i cannot solve it myself



Others already addressed your problem (cf Paul and Diez answers). I'll 
just allow myself to point a couple other potential problems with your code:



##worldmap module
class GeoMap:
cells = []
activerow = 0
activecol = 0



Attributes defined at the class level are *class* attributes, shared by 
all instances of the class - that is, all instances will access the one 
same 'cells' list. Instance attributes are canonically created in the 
initialize method (__init__) that is automagically called on 
instanciation. IOW, you want to replace the above lines with:


 def __init__(self):
 self.cells = []
 self.activerow = 0
 self.activecol = 0



def addCell(self, acell):
if len(self.cells) == 0:


An empty list evals to False in a boolean context, so the above can be 
simply expressed as:

  if not self.cells:


  self.cells.append([])
  self.activerow = 0
acell.col = self.activerow
acell.row = self.activecol
self.cells[self.activerow].append(acell)
self.activecol += 1

def addRow(self):
self.cells.append([])
self.activerow += 1;
self.activecol = 0;

class GeoMapCell:
neighbours = (None, None, None, None, None, None, )
col = 0
row = 0


Same remark as above. You want to move all this code to the __init__ method.

--
http://mail.python.org/mailman/listinfo/python-list

Re: is there a project running (GUI Builder for Python ) ?

[Hendrik van Rooyen]

"azrael"  wrote:


> To be honest, in compare to Visual Studio, Gui Builders for wx widgets
> are really bad. Also completly for python there is not one good
> GuiBuilder. The only one I have seen that would come near VS was
> BoaConstructor, But the number of Bugs is just horrific. Too bad that
> no one is developing it further.

What are these horrific bugs - I was forced to use Boa on a project
recently, and I did not notice any - In fact, I was impressed because
everything that was there, seemed to be working properly.

- Hendrik

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with objects copying each other in memory

[Bruno Desthuilliers]

Cameron Pulsford a écrit :

Thanks, that did it! Why is that the case though? Or rather, why do the 
assignments to temp.x and temp.y not effect the self.x and self.y? How 
come I only run into the problem with the list?



Because there's a huge difference between binding an object to a name 
and mutating an object ?


First point: Python's "assignment"  is really a binding of a name and an 
object _reference_ in a given namespace. Think of namespaces as 
name=>object_ref dictionnaries. This implies that "assignement" never 
copies anything. So when you do:


>>> list1 = []
>>> list2 = list1

the second line actually creates *another* name=>object pair referencing 
the *same* list object. IOW, list1 and list2 are two named references to 
a single object:


 >>> list1 is list2
True
>>> id(list1) == id(list2)
True

So whether you access it thru name 'list1' or 'list2', if you mutate the 
object (like append/remove/replace an element of the list), you'll see 
the result thru the other name as well:


>>> list1.append('foo')
>>> list2.append('bar')
>>> list1
['foo', 'bar']
>>> list2
['foo', 'bar']
>>>

Note FWIW that list subscripting (somelist[x] = y) is really a method 
call (somelist.__setitem__(x, y)) in disguise, so the same reasonning 
applies.


Now *rebinding* a name is a different thing. It makes the name refer to 
another object, but has no impact on other name=>object bindings 
refering to the previously bound object, ie:


>>> list2 = ['hou', 'lala']

Now list2 points to a newly created list object. This doesn't impact 
list1 of course:


>>> list1
['foo', 'bar']
>>> list1 is list2
False
>>> id(list1) == id(list2)
False
>>>


(snip)
--
http://mail.python.org/mailman/listinfo/python-list

Problem building Python extension

[martijnsteenwijk]

Hi all,

I'm trying to build my first python extensionon a win32 system. I
followed the description in 
http://starship.python.net/crew/mwh/toext/your-first-extension.html,
but after running

C:\Python26\python first-setup.py build_ext -i

nothing seems to happen and no file first.pyd is produced.

The result from the cmd is the following:
running build_ext
building `first` extension
error: None

Does anyone have an idea what's going wrong?

Thanks for replies in advance,
Martijn
--
http://mail.python.org/mailman/listinfo/python-list

Re: A little bit else I would like to discuss

[Marco Mariani]

azrael wrote:


I know that there is already a standard python library, But
why not extending it. classify the standard library into subcategories
like Networking, DataBase, Computation, ..


If the standard library where that huge, python 3.0 would have been late 
by a couple of years.



Why not using this number of people and accomplish something great. If
anyone of us would write 10 line of good code, it would result a very
great and powerfull environment.


I totally want to write my 10 lines of great image processing or speech 
recognition software, but it's not how development works.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem building Python extension

[David Cournapeau]

On Fri, Feb 13, 2009 at 6:30 PM,   wrote:
> Hi all,
>
> I'm trying to build my first python extensionon a win32 system. I
> followed the description in 
> http://starship.python.net/crew/mwh/toext/your-first-extension.html,
> but after running
>
> C:\Python26\python first-setup.py build_ext -i
>
> nothing seems to happen and no file first.pyd is produced.
>
> The result from the cmd is the following:
> running build_ext
> building `first` extension
> error: None

It is a bug in distutils - you reminded me that I did not report it :)
. IIRC, it is caused by the lack of a recognized installation VS 2008.
You should either install VS 2008 (the express vesion is free) or make
sure your VS 2008 is correctly installed.

Building python 2.6 extensions with mingw is possible as well, but I
think you need to build your own mingw (because python 2.6 requires
msvcr90.dll) - maybe recent mingw have msvcrt90.dll included.

cheers,

David
--
http://mail.python.org/mailman/listinfo/python-list

Re: something wrong with isinstance

[Carl Banks]

On Feb 12, 10:49 am, redbaron  wrote:
> Don't really sure, but try to define your class as new-style one.
> Like
> class GeoMap(object):
>    ...

Well, the OP said he was using Python 3.0, where all classes are new-
style classes.

But that brings up another very slight possibility, though not a very
likely one in this case: the behavior of isinstance can be
customized.  It can happen unbeknownst to a user who subclasses a
class that does that.


Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list

Levenshtein word comparison -performance issue

[S.Selvam Siva]

Hi all,

I need some help.
I tried to find top n(eg. 5) similar words for a given word, from a
dictionary of 50,000 words.
I used python-levenshtein module,and sample code is as follow.

def foo(searchword):
disdict={}
for word in self.dictionary-words:
   distance=Levenshtein.ratio(searchword,word)
   disdict[word]=distance
"""
 sort the disdict dictionary by values in descending order
"""
similarwords=sorted(disdict, key=disdict.__getitem__, reverse=True)

return similarwords[:5]

foo() takes a search word and compares it with dictionary of 50,000 and
assigns each word a value(lies between 0 to 1).
Then after sorting in descending order it returns top 5 similar words.

The problem is, it* takes long time* for processing(as i need to pass more
search words within a loop),i guess the code could be improved to work
efficiently.Your suggestions are welcome...
-- 
Yours,
S.Selvam
--
http://mail.python.org/mailman/listinfo/python-list

Re: Break large file down into multiple files

[Gabriel Genellina]

En Fri, 13 Feb 2009 05:43:02 -0200, brianrpsgt1   
escribió:



On Feb 12, 11:02 pm, "Gabriel Genellina" 
wrote:

En Fri, 13 Feb 2009 04:44:54 -0200, brianrpsgt1   
escribió:

> New to python I have a large file that I need to break upinto
> multiple smallerfiles. I need to break the large fileintosections
> where there are 65535 lines and then write thosesectionsto seperate
>files.  I am familiar with opening and writingfiles, however, I am
> struggling with creating thesectionsand writing the different
>sectionsto their ownfiles.

This function copies at most n lines from fin to fout:

def copylines(fin, fout, n):
   for i, line in enumerate(fin):
     fout.write(line)
     if i+1>=n: break

Now you have to open the source file, create newfilesas needed and  
repeatedly call the above function until the end of source file. You'll  
 

have to enhace it bit, to know whether there are remaining lines or not.

--
Gabriel Genellina


Gabriel ::

Thanks for the direction.  Do I simply define fin, fout and n as
variables above the def copylines(fin, fout, n): line?

Would it look like this?

fin = open('C:\Path\file')
fout = 'C:\newfile.csv')
n = 65535


Warning: see this FAQ entry  
http://www.python.org/doc/faq/general/#why-can-t-raw-strings-r-strings-end-with-a-backslash


def copylines(fin, fout, n):
for i, line in enumerate(fin):
      fout.write(line)
      if i+1>=n: break


Almost. You have to *call* the copylines function, not just define it.  
After calling it with: copylines(fin, fout, 65535), you'll have the  
*first* chunk of lines copied. So you'll need to create a second file,  
call the copylines function again, create a third file, call... You'll  
need some kind of loop, and a way to detect when to stop and break out of  
it.


The copylines function already knows what happened (whether there are more  
lines to copy or not) so you should enhace it and return such information  
to the caller.


It isn't so hard, after working out the tutorial (linked from  
http://wiki.python.org/moin/BeginnersGuide ) you'll know enough Python to  
finish this program.


If you have some previous programming experience, Dive into Python (linked  
from the Beginners Guide above) is a good online book.


Feel free to come back when you're stuck with something.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: something wrong with isinstance

[Gabriel Genellina]

En Fri, 13 Feb 2009 08:07:58 -0200, Carl Banks   
escribió:



Well, the OP said he was using Python 3.0, where all classes are new-
style classes.

But that brings up another very slight possibility, though not a very
likely one in this case: the behavior of isinstance can be
customized.  It can happen unbeknownst to a user who subclasses a
class that does that.


Really? I didn't know that -- how do you do that?

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: thread. question

[Carl Banks]

On Feb 9, 7:34 am, Tim Wintle  wrote:
> Thanks for both replies,
>
> On Mon, 2009-02-09 at 15:59 +0100, Christian Heimes wrote:
> > You shouldn't use the thread module directly. It's not meant to be used
> > by a user. Please stick to the threading module. You won't notice a
> > slowdown, trust me :)
>
> I'm aware that thread is being renamed to _thread in python 3.0, but is
> it being depricated or anything like that?
>
> This is for an app that has been running for quite a long time and it's
> now time for fairly heavy optimisations as load is increasing (Believe
> me, I wouldn't have been looking at the C otherwise) - so I'll see if I
> do notice any effect with threading.

The threading module is likely to be slightly slower when spawning new
threads, because the Thread class has a bit of overhead that allows
you to do things like thread.join().

Simple locking will be same speed whether you use the threading or
thread module, because threading.Lock and thread.acquire return the
same type of object.  This is an implementation detail, though.


Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list

Re: Levenshtein word comparison -performance issue

[Gabriel Genellina]

En Fri, 13 Feb 2009 08:16:00 -0200, S.Selvam Siva   
escribió:



I need some help.
I tried to find top n(eg. 5) similar words for a given word, from a
dictionary of 50,000 words.
I used python-levenshtein module,and sample code is as follow.

def foo(searchword):
disdict={}
for word in self.dictionary-words:
   distance=Levenshtein.ratio(searchword,word)
   disdict[word]=distance
"""
 sort the disdict dictionary by values in descending order
"""
similarwords=sorted(disdict, key=disdict.__getitem__, reverse=True)

return similarwords[:5]


You may replace the last steps (sort + slice top 5) by heapq.nlargest - at  
least you won't waste time sorting 49995 irrelevant words...
Anyway you should measure the time taken by the first part (Levenshtein),  
it may be the most demanding. I think there is a C extension for this,  
should be much faster than pure Python calculations.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: something wrong with isinstance

[Peter Otten]

Gabriel Genellina wrote:

> En Fri, 13 Feb 2009 08:07:58 -0200, Carl Banks 
> escribió:
> 
>> Well, the OP said he was using Python 3.0, where all classes are new-
>> style classes.
>>
>> But that brings up another very slight possibility, though not a very
>> likely one in this case: the behavior of isinstance can be
>> customized.  It can happen unbeknownst to a user who subclasses a
>> class that does that.
> 
> Really? I didn't know that -- how do you do that?
 
>>> class Type(type):
... def __instancecheck__(self, other): return True
...
>>> class A(metaclass=Type): pass
...
>>> class B: pass
...
>>> isinstance(B(), A)
True

See also 

http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubclass

I doubt that this is the source of the OP's troubles, though.

Peter
--
http://mail.python.org/mailman/listinfo/python-list

Re: something wrong with isinstance

[Gabriel Genellina]

En Fri, 13 Feb 2009 08:43:03 -0200, Peter Otten <__pete...@web.de>  
escribió:

Gabriel Genellina wrote:
En Fri, 13 Feb 2009 08:07:58 -0200, Carl Banks  


escribió:

But that brings up another very slight possibility, though not a very
likely one in this case: the behavior of isinstance can be
customized.  It can happen unbeknownst to a user who subclasses a
class that does that.


Really? I didn't know that -- how do you do that?



class Type(type):

... def __instancecheck__(self, other): return True
...

class A(metaclass=Type): pass

...

class B: pass

...

isinstance(B(), A)

True

See also

http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubclass


Ah, ok. Isn't menctioned in the main documentation though.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem building Python extension

[martijnsteenwijk]

On 13 feb, 10:53, David Cournapeau  wrote:
> On Fri, Feb 13, 2009 at 6:30 PM,   wrote:
> > Hi all,
>
> > I'm trying to build my first python extensionon a win32 system. I
> > followed the description 
> > inhttp://starship.python.net/crew/mwh/toext/your-first-extension.html,
> > but after running
>
> > C:\Python26\python first-setup.py build_ext -i
>
> > nothing seems to happen and no file first.pyd is produced.
>
> > The result from the cmd is the following:
> > running build_ext
> > building `first` extension
> > error: None
>
> It is a bug in distutils - you reminded me that I did not report it :)
> . IIRC, it is caused by the lack of a recognized installation VS 2008.
> You should either install VS 2008 (the express vesion is free) or make
> sure your VS 2008 is correctly installed.
>
> Building python 2.6 extensions with mingw is possible as well, but I
> think you need to build your own mingw (because python 2.6 requires
> msvcr90.dll) - maybe recent mingw have msvcrt90.dll included.
>
> cheers,
>
> David

Thanks a lot for your reply. I downloaded & installed Visual C# 2008
express, but unfortunately this doesn't change anything in running the
setup file. Unfortunately, still no pyd file is produced...

Did I something wrong?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Untangling pythonWin and IDLE Processes on XP Pro

[W. eWatson]

Terry Reedy wrote:

W. eWatson wrote:

 From Diez above.
What does *NOT* work is writing a Tkinter-based app in idle, and to 
run it

*FROM INSIDE* idle. Instead, open your explorer and double-click on the
pyhton-file your app is in. That's all that there is to it.

So this is the absolute truth? No wiggle room? One can never use a 
Tkinter program with IDLE, and execute it successfully. So IDLE 
doesn't issue a standard warning that says, "Get out of here with your 
Tkinter program, it will fail when you try to run it here. You have 
entered Tkinter hell. Good-bye."


Re-read my post about kids fighting to control a television.  Maybe they 
work together, maybe they crash the TV.  Hard to predict.


***ANY*** Python program that tries to grab and control the same 
resources that TK does may conflict with it.  There is no way that IDLE 
can have a list of, for instance, all event-grabbing mainloop programs.


OK, enough tinkering with the code and others matters on my end trying to 
find a work around. Somehow after much successful use of IDLE's execution 
facility, I've stepped on an invisible banana peel. I think it's evident 
that I'm not going around this problem easily with the IDLE execution 
attempts, and that another solution is required.


First, I think somewhere up the thread someone suggested that Active 
pythonWin is not dependent upon Tk, correct? Therefore, it is immune from 
such problems, correct?


Second, maybe I missed it above, but when I posted the output from the 
program that showed the failure, was there anything that said, "IDLE 
problem" or would even give a clue that's the culprit?


Finally, we can probably agree that I can continue to use IDLE for editing 
and syntax checking, but to "guarantee" successful execution of the program, 
I can just double-click on the py file in my folder. Perhaps there is a 
better way than clicking on it in the folder. For example, putting it on the 
desktop. As I look at the folder, previous copies only differ by a digit, I 
can easily find myself executing an earlier version, differing as Dev4, to 
Dev5 at the end of each name.


Let me ask this. When I install Active Python, am I getting something beyond 
their interface? That is, does executing the code there result in using the 
same python interpreter that is used by IDLE? My use of their editor has 
been somewhat exasperating. It does not seem as friendly as the IDLE editor.


I still find it bizarre that the original creator of this program can spend 
months using IDLE to develop this program, and that I've spent maybe 10 days 
recently now adding to it without having much, if any, problem with IDLE and 
the programs execution within IDLE. I asked him almost a year ago what tool 
he used. IDLE, was the reply. Maybe it was really IDLE with no execution 
from inside IDLE. I'll ask him.


--
   W. eWatson

 (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
  Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet

Web Page: 

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem building Python extension

[David Cournapeau]

On Fri, Feb 13, 2009 at 7:58 PM,   wrote:
> On 13 feb, 10:53, David Cournapeau  wrote:
>> On Fri, Feb 13, 2009 at 6:30 PM,   wrote:
>> > Hi all,
>>
>> > I'm trying to build my first python extensionon a win32 system. I
>> > followed the description 
>> > inhttp://starship.python.net/crew/mwh/toext/your-first-extension.html,
>> > but after running
>>
>> > C:\Python26\python first-setup.py build_ext -i
>>
>> > nothing seems to happen and no file first.pyd is produced.
>>
>> > The result from the cmd is the following:
>> > running build_ext
>> > building `first` extension
>> > error: None
>>
>> It is a bug in distutils - you reminded me that I did not report it :)
>> . IIRC, it is caused by the lack of a recognized installation VS 2008.
>> You should either install VS 2008 (the express vesion is free) or make
>> sure your VS 2008 is correctly installed.
>>
>> Building python 2.6 extensions with mingw is possible as well, but I
>> think you need to build your own mingw (because python 2.6 requires
>> msvcr90.dll) - maybe recent mingw have msvcrt90.dll included.
>>
>> cheers,
>>
>> David
>
> Thanks a lot for your reply. I downloaded & installed Visual C# 2008
> express, but unfortunately this doesn't change anything in running the
> setup file. Unfortunately, still no pyd file is produced...

I am not 100 % confident, but I think the problem is that for Visual
Studio express, you need to build the extension in a Visual Studio
shell (in the VS 2008 express menu). If that still does not work, I
will check on a windows VM (I need to do the bug report anyway).

cheers,

David
--
http://mail.python.org/mailman/listinfo/python-list

Re: Invoking CutePDF from within Python

[cm]

Hi John,

All I need is to say "Print this to CUTEPDF and store as xyz.pdf".
I can't answer you question but let me make a suggestion: Try 
PdfCreator. It lets you control all the process using an activex 
control. It has events to tell you when the jobs has finish, or report 
you of eventual errors.


Download it from sf.net, and look into the samples.

Best regards,

Carlos.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Break large file down into multiple files

[Chris]

On Feb 13, 10:02 am, redbaron  wrote:
> > New to python I have a large file that I need to break up into
> > multiple smaller files. I need to break the large file into sections
> > where there are 65535 lines and then write those sections to seperate
> > files.
>
> If your lines are variable-length, then look at itertools recipes.
>
> from itertools import izip_longest
>
> def grouper(n, iterable, fillvalue=None):
>     "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
>     args = [iter(iterable)] * n
>     return izip_longest(fillvalue=fillvalue, *args)
>
> with open("/file","r") as f:
>     for lines in grouper(65535,f,""):
>         data_to_write = '\n'.join(lines).rstrip("\n")
>         ...
>         
>         ...

I really would not recommend joining a large about of lines, that will
take some times.

fIn = open(input_filename, 'rb')
chunk_size = 65535

for i,line in enumerate(fIn):
if not i:   # First Line in the File, create a file to start
writing to
filenum = '%04d'%(i%chunk_size)+1
fOut = open('%s.txt'%filenum, 'wb')
if i and not i % chunk_size:   # Once at the chunk_size close the
old file object and create a new one
fOut.close()
filenum = '%04d'%(i%chunk_size)+1
fOut = open('%s.txt'%filenum, 'wb')
if not i % 1000:
fOut.flush()
fOut.write(line)

fOut.close()
fIn.close()
--
http://mail.python.org/mailman/listinfo/python-list

Re: Change in cgi module's handling of POST requests

[Gabriel Genellina]

En Fri, 13 Feb 2009 02:43:48 -0200, Bob Kline   
escribió:

Joshua Kugler wrote:
We just upgraded Python to 2.6 on some of our servers and a number of  
our

CGI scripts broke because the cgi module has changed the way it handles
POST requests.  When the 'action' attribute was not present in the  
form element on an HTML page the module behaved as if the value of the

attribute was the URL which brought the user to the page with the form,
but without the query (?x=y...) part.


[1] I haven't yet finished my attempts to parse the relevant RFCs; I  
assumed that the original authors and maintainers of this module (which  
includes the BDFL himself), would have been more adept at that than I  
am, which is one of the reasons I was hoping to find some discussion in  
the mailing list archives of the discussion of the proposed change in  
the module's behavior.


I noticed this change in behaviour too, and indeed, it is due to  
http://bugs.python.org/issue1817
But I could not find any RFC/standard/reccomendation/whatever that clearly  
states *what* should happen with a POST request directed to an URI having  
a query string itself. So I could not say "it's a bug" -- it's just...  
annoying in these cases (although it might be useful in some other cases,  
I think)
A posible workaround is to ensure all forms have an action attribute, and  
that it doesn't contain any query string.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Break large file down into multiple files

[Tim Chase]

New to python I have a large file that I need to break up
into multiple smaller files. I need to break the large file
into sections where there are 65535 lines and then write those
sections to seperate files.  I am familiar with opening and
writing files, however, I am struggling with creating the
sections and writing the different sections to their own
files.


While this thread has offered many nice Python solutions, the 
"split" command is pretty standard on most Linux boxes:


  bash$ split -l 65535 infile.txt

which will do what you describe.  You can read the man page for 
more details.


So if you just want a fast route to a goal rather than going 
through the learning process (I'm all for learning how to do it 
too), this may be a quick answer.


-tkc



--
http://mail.python.org/mailman/listinfo/python-list

Re: Break large file down into multiple files

[Chris]

On Feb 13, 1:19 pm, Chris  wrote:
> On Feb 13, 10:02 am, redbaron  wrote:
>
>
>
> > > New to python I have a large file that I need to break up into
> > > multiple smaller files. I need to break the large file into sections
> > > where there are 65535 lines and then write those sections to seperate
> > > files.
>
> > If your lines are variable-length, then look at itertools recipes.
>
> > from itertools import izip_longest
>
> > def grouper(n, iterable, fillvalue=None):
> >     "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
> >     args = [iter(iterable)] * n
> >     return izip_longest(fillvalue=fillvalue, *args)
>
> > with open("/file","r") as f:
> >     for lines in grouper(65535,f,""):
> >         data_to_write = '\n'.join(lines).rstrip("\n")
> >         ...
> >         
> >         ...
>
> I really would not recommend joining a large about of lines, that will
> take some times.
>
> fIn = open(input_filename, 'rb')
> chunk_size = 65535
>
> for i,line in enumerate(fIn):
>     if not i:   # First Line in the File, create a file to start
> writing to
>         filenum = '%04d'%(i%chunk_size)+1
>         fOut = open('%s.txt'%filenum, 'wb')
>     if i and not i % chunk_size:   # Once at the chunk_size close the
> old file object and create a new one
>         fOut.close()
>         filenum = '%04d'%(i%chunk_size)+1
>         fOut = open('%s.txt'%filenum, 'wb')
>     if not i % 1000:
>         fOut.flush()
>     fOut.write(line)
>
> fOut.close()
> fIn.close()

Whoops, day-dreaming mistake.  Use "filenum = '%04d'%(i/chunk_size)+1"
and not i%chunk_size.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem building Python extension

[Christian Heimes]

martijnsteenw...@gmail.com wrote:
> Thanks a lot for your reply. I downloaded & installed Visual C# 2008
> express, but unfortunately this doesn't change anything in running the
> setup file. Unfortunately, still no pyd file is produced...
> 
> Did I something wrong?

Yeah, you installed the C# environment. Python is written in C, so you
have to get Visual Studio C++ 2008. However a recent version of MinGW32
is sufficient to build most extensions.

Christian

--
http://mail.python.org/mailman/listinfo/python-list

get wget log message

[JuergenRiemer]

Hi,

I didn't succeed to do the following:
I download a PDF file via wget yet I am not interested in the file
itself but the wget verbose output (log message) e.g. download speed,
total size, etc.
I know I could use < -o log.txt > and then read from that file yet
this seems not very elegant.
I am new to both wget/linux and python and am struggling for quite
some time now, could anyone help me here?

 thx a lot
 Juergen

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem building Python extension

[Giampaolo Rodola']

On Feb 13, 10:53 am, David Cournapeau  wrote:
> On Fri, Feb 13, 2009 at 6:30 PM,   wrote:
> > Hi all,
>
> > I'm trying to build my first python extensionon a win32 system. I
> > followed the description 
> > inhttp://starship.python.net/crew/mwh/toext/your-first-extension.html,
> > but after running
>
> > C:\Python26\python first-setup.py build_ext -i
>
> > nothing seems to happen and no file first.pyd is produced.
>
> > The result from the cmd is the following:
> > running build_ext
> > building `first` extension
> > error: None
>
> It is a bug in distutils - you reminded me that I did not report it :)
> . IIRC, it is caused by the lack of a recognized installation VS 2008.
> You should either install VS 2008 (the express vesion is free) or make
> sure your VS 2008 is correctly installed.
>
> Building python 2.6 extensions with mingw is possible as well, but I
> think you need to build your own mingw (because python 2.6 requires
> msvcr90.dll) - maybe recent mingw have msvcrt90.dll included.
>
> cheers,
>
> David

I already reported that:
http://bugs.python.org/issue4931


--- Giampaolo
http://code.google.com/p/pyftpdlib
--
http://mail.python.org/mailman/listinfo/python-list

Re: get wget log message

[JuergenRiemer]

> itself but the wget verbose output (log message) e.g. download speed,
> total size, etc.

ahh I got it 1 minute after sending this post g

commands.getstatusoutput(cmd)

Juergen
--
http://mail.python.org/mailman/listinfo/python-list

Re: Change in cgi module's handling of POST requests

[Paul Boddie]

On 13 Feb, 11:58, "Gabriel Genellina"  wrote:
>
> I noticed this change in behaviour too, and indeed, it is due to  
> http://bugs.python.org/issue1817
> But I could not find any RFC/standard/reccomendation/whatever that clearly  
> states *what* should happen with a POST request directed to an URI having  
> a query string itself. So I could not say "it's a bug" -- it's just...  
> annoying in these cases (although it might be useful in some other cases,  
> I think)

The issue of distinguishing between query-originating parameters and
form-originating parameters has been around for a very long time, and
in my own work, especially where the cgi module has been used, I've
been careful to distinguish between the two types and to merge them
only if this is desired (get_fields vs. get_fields_from_path and
get_fields_from_body in WebStack). If the cgi module has now changed
its behaviour to just merge the two data sources, I can imagine that a
few applications aren't going to be happy any more, and since I doubt
that I'm the only one to take the approach I've just described, I can
imagine that I'm not alone in regarding this "bug fix" as more of a
regression if it has been done in a way which does not preserve the
behaviour of unmodified existing code.

Paul
--
http://mail.python.org/mailman/listinfo/python-list

Re: A little bit else I would like to discuss

[Martin]

2009/2/12 Christian Heimes :
> Martin wrote:
> [typos igored as requested ;)]
>
>> How does "small and agile" work with "batteries included"?
>
> The Python slogan says "batteries included", not "fusion reactor included".

I'd be fine with a fusion reactor, my objections would be if skynet
was included :)

> The rules are:
>
> ...
>
> It takes at least 1.5 years to get a new feature into an extension and
> at least 3 years to remove or change a feature. That's a major shop
> stopper for every fast moving piece of Python software.

True, I thought a bit more about it and decided for myself to be happy
with the way it is since I prefer quality to features :)


/Martin




-- 
http://soup.alt.delete.co.at
http://www.xing.com/profile/Martin_Marcher
http://www.linkedin.com/in/martinmarcher

You are not free to read this message,
by doing so, you have violated my licence
and are required to urinate publicly. Thank you.

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html
--
http://mail.python.org/mailman/listinfo/python-list

Re: NNTPlib encoding issue

[MRAB]

mohit_ranka wrote:

I am getting unknown encoding like, "=?Utf-8?B?QWRyaWFu?=
" (quotes for clarity) for name of
the author from nntplib module. which seems like an encoding issue
with NNTPLib module.

What should i do to get author name information in human readable
format.


The original text was encoded in UTF-8 and then into base-64:

=?Utf-8?B?QWRyaWFu?=
  ^ UTF-8

=?Utf-8?B?QWRyaWFu?=
^ Base-64

So that's:

>>> "QWRyaWFu".decode("base-64").decode("Utf-8")
u'Adrian'
--
http://mail.python.org/mailman/listinfo/python-list

Re: An executable operational semantics for Python

[gideon]

On Feb 12, 5:14 pm, bearophileh...@lycos.com wrote:
> gideon:
>
> > I've recently finished my Master's thesis on the semantics of Python.
> > In my thesis I define the semantics of Python by rewriting an abstract
> > machine. The sources that are used to produce my thesis can also be
> > compiled into a working interpreter. Hence I call it an 'executable'
> >semantics.
>
> Can it be used for some useful purpose?

The interpreter? Probably not. The semantics. Yes.
--
http://mail.python.org/mailman/listinfo/python-list

best set of modules for web automation without javascript

[News123]

Hi,


I'd like to do some web automation with python 2.5
- https:
- a cookiejar
- some forms to be filled in


what is the best set of modules.

As far as I understood, there is httplib, but it seems (if I understood
well) to be incoompatible with cookielib

I'm a newcomer to webautomation with python and would be thankful for
good suggestions.


I used so far perl with LWP::UserAgent HTTP::Cookies and some module (I
forgot the name) to handle parsing and filling in Forms.

thanks in advance for any pointers opinions


N
--
http://mail.python.org/mailman/listinfo/python-list

sgmllib parser keeps old tag data?

[Berend van Berkum]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Hi everyone,

I read the source, made numerous tests, but SGMLParser's keeps returning *tag* 
data 
from previous parser instances. I'm totally confused why.. The content data it
returns is ok.

E.g.::

sp = MyParser()
sp.feed('Test')
print sp.content, sp.markup
sp.close()

sp = MyParser()
sp.feed('\n\r\n')
print sp.content, sp.markup
sp.close()

gives::

('Test', [{'t': ({}, (0, 0))}, {'test': ({}, (0, 4))}]) 
('\n\r\n', [{'t': ({}, (0, 0))}, {'test': ({}, (0, 4))}, {'xml': ({}, (0, 
1))}])

It keeps the tags from the previous session, while i'm sure the stack etc.
should be clean..

Any ideas?


regards, Berend

- 

import sgmllib


class MyParser(sgmllib.SGMLParser):

content = ''
markup = []
span_stack = []

def handle_data(self, data):
self.content += data

def unknown_starttag(self, tag, attr):
stack = { tag: ( dict(attr), ( len(self.content), ) ) }
self.span_stack.append(stack)

def unknown_endtag(self, tag):
prev_tag, ( attr, ( offset, ) ) = 
self.span_stack.pop().items()[0]

if tag:
# close all tags on stack until it finds a matching end 
tag
# XXX: need to return to LEVEL, not same tag name
while tag != prev_tag:
span = { prev_tag: ( attr, ( offset, 0 ) ) }
self.markup.append( span )

prev_tag, ( attr, ( offset, ) ) = 
self.span_stack.pop().items()[0]

length = len( self.content ) - offset
span = { tag: ( attr, ( offset, length ) ) }
self.markup.append( span )

def do_unknown_tag(self, tag, attr):
assert not tag and not attr, "do_unknown_tag %s, %s" % (tag, 
attr)

def close(self):
sgmllib.SGMLParser.close(self)
self.content = ''
self.markup = []
self.span_stack = []


def parse_data(data):
sp = MyParser()
sp.feed(data)
r = sp.content, sp.markup
sp.close()
return r

print parse_data('Test')
print parse_data('\n\r\n')
print parse_data('Test 3')



- -- 
 web, http://dotmpe.com  ()ASCII Ribbon
 email, berend.van.ber...@gmail.com  /\
 icq, 26727647;  irc, berend/mpe at irc.oftc.net

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFJlXxrn70fkTNDJRgRArWwAKCbhe/FwOu3/XtAja7+rbvIv29HEQCgwtf3
k3eiwfD0yw6t+giXJy1nako=
=afE6
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list

Re: best set of modules for web automation without javascript

[Joe Riopel]

On Fri, Feb 13, 2009 at 9:04 AM, News123  wrote:
> Hi,
> I'd like to do some web automation with python 2.5
> - https:
> - a cookiejar
> - some forms to be filled in
> what is the best set of modules.

I have automated some testing of our product, using it's web UI with
Python with urllib2 and urrlib. I don't actually fill in the forms, I
just recreate the post and set the values of the post variables (so I
don't get any form validation).

Check out:
urllib2.build_opener
urllib2.HTTPCookieProcessor
urllib2.Request
--
http://mail.python.org/mailman/listinfo/python-list

Re: sgmllib parser keeps old tag data?

[MRAB]

Berend van Berkum wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Hi everyone,

I read the source, made numerous tests, but SGMLParser's keeps returning *tag* data 
from previous parser instances. I'm totally confused why.. The content data it

returns is ok.

E.g.::

sp = MyParser()
sp.feed('Test')
print sp.content, sp.markup
sp.close()

sp = MyParser()
sp.feed('\n\r\n')
print sp.content, sp.markup
sp.close()

gives::

('Test', [{'t': ({}, (0, 0))}, {'test': ({}, (0, 4))}]) 
('\n\r\n', [{'t': ({}, (0, 0))}, {'test': ({}, (0, 4))}, {'xml': ({}, (0, 1))}])


It keeps the tags from the previous session, while i'm sure the stack etc.
should be clean..

Any ideas?


regards, Berend

- 

import sgmllib


class MyParser(sgmllib.SGMLParser):

content = ''
markup = []
span_stack = []


These are in the _class_ itself, so they will be shared by all its
instances. You should so something like this instead:

def __init__(self):
self.content = ''
self.markup = []
self.span_stack = []


def handle_data(self, data):
self.content += data

def unknown_starttag(self, tag, attr):
stack = { tag: ( dict(attr), ( len(self.content), ) ) }
self.span_stack.append(stack)

def unknown_endtag(self, tag):
prev_tag, ( attr, ( offset, ) ) = 
self.span_stack.pop().items()[0]

if tag:
# close all tags on stack until it finds a matching end 
tag
# XXX: need to return to LEVEL, not same tag name
while tag != prev_tag:
span = { prev_tag: ( attr, ( offset, 0 ) ) }
self.markup.append( span )

prev_tag, ( attr, ( offset, ) ) = 
self.span_stack.pop().items()[0]

length = len( self.content ) - offset
span = { tag: ( attr, ( offset, length ) ) }
self.markup.append( span )

def do_unknown_tag(self, tag, attr):
assert not tag and not attr, "do_unknown_tag %s, %s" % (tag, 
attr)

def close(self):
sgmllib.SGMLParser.close(self)
self.content = ''
self.markup = []
self.span_stack = []


def parse_data(data):
sp = MyParser()
sp.feed(data)
r = sp.content, sp.markup
sp.close()
return r

print parse_data('Test')
print parse_data('\n\r\n')
print parse_data('Test 3')



--
http://mail.python.org/mailman/listinfo/python-list

Re: Invoking CutePDF from within Python

[python]

Tim,

> FWIW, I usually generate PDFs by printing to a Postscript printer (some 
> random Apple Laserthingy) and then using Ghostscript to do the conversion. 
> I'm happy to post if you're interested.

If its not too much bother, I would be interested in learning how you're
interfacing to Python to Ghostscript.

Regards,
Malcolm
--
http://mail.python.org/mailman/listinfo/python-list

Re: Invoking CutePDF from within Python

[Tim Golden]

pyt...@bdurham.com wrote:

Tim,


FWIW, I usually generate PDFs by printing to a Postscript printer (some random 
Apple Laserthingy) and then using Ghostscript to do the conversion. I'm happy 
to post if you're interested.


If its not too much bother, I would be interested in learning how you're
interfacing to Python to Ghostscript.


Really boring, I'm afraid. Just popen and an .exe:

http://pastebin.com/m461bf8f2

(There are a few dependencies I haven't shown but I
hope the approach is obvious; happy to explain / supply
details)

TJG
--
http://mail.python.org/mailman/listinfo/python-list

ANN: Supy 1.1

[Greg Ewing]

SuPy 1.1 Available
--

   http://www.cosc.canterbury.ac.nz/greg.ewing/SuPy/

Changes in this version:

- Added explicit ways of calling method names ending in '?' or '!'. Python
method names 'is_xxx' and 'xxx_ip' map to Ruby 'xxx?' and 'xxx!' respectively.
The plain name 'xxx' can still be used where there is no ambiguity.

- Ruby true and false are now converted to Python True and False.

- Ruby methods expecting a block can be called from Python by passing
a callable Python object with the keyword 'body'.


What is SuPy?
-

SuPy is a plugin for the Sketchup 3D modelling application
that lets you script it in Python.

--
Greg Ewing
greg.ew...@canterbury.ac.nz
--
http://mail.python.org/mailman/listinfo/python-list

Re: sgmllib parser keeps old tag data?

[Berend van Berkum]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, Feb 13, 2009 at 02:31:40PM +, MRAB wrote:
> Berend van Berkum wrote:
> >
> >import sgmllib
> >
> >
> >class MyParser(sgmllib.SGMLParser):
> >
> > content = ''
> > markup = []
> > span_stack = []
> >
> These are in the _class_ itself, so they will be shared by all its
> instances. You should so something like this instead:
> 
>   def __init__(self):
>   self.content = ''
>   self.markup = []
>   self.span_stack = []
> 

Yes.. tested that and SGMLParser won't let me override __init__, 
(SGMLParser vars are uninitialized even with sgmllib.SGMLParser(self) call).
Tried some but not the following:
with a differently named init function and one boolean class var 'initialized'
it can check 'if self.initialized' in front of each handler. Does the trick.

Confusion dissolved :)
thanks.

- -- 
 web, http://dotmpe.com  ()ASCII Ribbon
 email, berend.van.ber...@gmail.com  /\
 icq, 26727647;  irc, berend/mpe at irc.oftc.net

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFJlYjVn70fkTNDJRgRAhFRAJ9XDPaR2zb8EjKfTACDjtzwI7z/9ACgzcmB
Ms1QZ9IoB2s6RJ+tdXJtzfs=
=itBb
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list

Re: Invoking CutePDF from within Python

[python]

Tim,

> http://pastebin.com/m461bf8f2

Wonderful! That's enough to get me started. Thank you very much.

Cheers,
Malcolm
--
http://mail.python.org/mailman/listinfo/python-list

Re: best set of modules for web automation without javascript

[News123]

Hi Joel,

Thanks,
This (the urllib2 methods you combined with cookielib) is what I am
currently trying to do.

I would just like to retrieve all the field names and default values of
a form. (Some forms are huge) and wondered thus whether there's already
a python module parsing a html documents for forms , form fields and
field vaules, returning an objcet. that could be modified and posted.


bye

N



Joe Riopel wrote:
> On Fri, Feb 13, 2009 at 9:04 AM, News123  wrote:
>> Hi,
>> I'd like to do some web automation with python 2.5
>> - https:
>> - a cookiejar
>> - some forms to be filled in
>> what is the best set of modules.
> 
> I have automated some testing of our product, using it's web UI with
> Python with urllib2 and urrlib. I don't actually fill in the forms, I
> just recreate the post and set the values of the post variables (so I
> don't get any form validation).
> 
> Check out:
> urllib2.build_opener
> urllib2.HTTPCookieProcessor
> urllib2.Request
--
http://mail.python.org/mailman/listinfo/python-list

Re: best set of modules for web automation without javascript

[Marco Mariani]

News123 wrote:


I would just like to retrieve all the field names and default values of
a form. (Some forms are huge) and wondered thus whether there's already
a python module parsing a html documents for forms , form fields and
field vaules, returning an objcet. that could be modified and posted.


http://wwwsearch.sourceforge.net/ClientForm/
http://wwwsearch.sourceforge.net/mechanize/
--
http://mail.python.org/mailman/listinfo/python-list

Re: [ANN] TracShell 0.1 released

[J Kenneth King]

J Kenneth King  writes:

> I tend to work a lot with Trac for project management and have always
> found the browser interface to be a productivity killer. I always
> wanted a simple command-line interface to Trac, but having never found
> one I found a little free time and got off my laurels to make one.
>
> TracShell 0.1 is an early release, but it works. So far you can only
> query and view tickets, but planned updates include the obvious
> ability to create and edit tickets. Future plans will allow browsing
> of comments, change histories, attachments, and so forth.
>
> Please consider it really beta. The code needs a little tidying up
> around the edges. If you find any bugs, please report them and I'll
> fix them ASAP. Ideas, suggestions, and contributions are welcome.
>
> http://code.google.com/p/tracshell/
>
> Cheers.

Just added the ability to create tickets, more features forthcoming.

I highly recommend anyone using this tool to stick to the latest svn
versions. I'll package the feature-complete stables going forward.

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: is there a project running (GUI Builder for Python ) ?

[J Kenneth King]

gc_ott...@yahoo.ca writes:

> ..I come from Delphi, and compared to Delphi, even Visual Studio
>> vanishes ;-)
> ...I don't even notice the difference between Delphi (which
> I'm still using)
>> and wxPython.
>>
>> I think this story happened to other people to,
>> so instead of putting a lot of effort in designing and maintaining a GUI
>> builders,
>> it might be better to choose another solution.
>>
>> btw, the idea I used, can be seen 
>> here]http://mientki.ruhosting.nl/data_www/pylab_works/pw_gui_support.html
>> and the code can be found 
>> herehttp://code.google.com/p/pylab-works/downloads/list
>>
>> cheers,
>> Stef
>
> You know, 10 or more years ago both Borland and Microsoft got it right
> when they incorporated a GUI with an IDE in their Delphi and Visual
> Basic products. As wonderful as the Python language is, it is very
> much a work in progress when compared to the ease of use of the
> aforementioned products. These products revolutionized the industry
> with their "Rapid Applications Development" (RAD).
>
> Python reminds me of a toolbox filled with a multitude of single use
> tools when all you need is a Skilsaw and a combination screwdriver.
> Gord

... So use the combination screwdriver.

Python isn't all things to all people. It is what it is.
--
http://mail.python.org/mailman/listinfo/python-list

Re: sgmllib parser keeps old tag data?

[MRAB]

Berend van Berkum wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, Feb 13, 2009 at 02:31:40PM +, MRAB wrote:

Berend van Berkum wrote:

import sgmllib


class MyParser(sgmllib.SGMLParser):

content = ''
markup = []
span_stack = []


These are in the _class_ itself, so they will be shared by all its
instances. You should so something like this instead:

def __init__(self):
self.content = ''
self.markup = []
self.span_stack = []



Yes.. tested that and SGMLParser won't let me override __init__, 
(SGMLParser vars are uninitialized even with sgmllib.SGMLParser(self) call).


OK, so SGMLParser needs to be initialised:

def __init__(self):
sgmllib.SGMLParser.__init__(self)
self.content = ''
self.markup = []
self.span_stack = []


Tried some but not the following:
with a differently named init function and one boolean class var 'initialized'
it can check 'if self.initialized' in front of each handler. Does the trick.

Confusion dissolved :)
thanks.


--
http://mail.python.org/mailman/listinfo/python-list

Re: sgmllib parser keeps old tag data?

[andrew cooke]

you are declaring class variables, not instance variables.  you need to
declare these in an __init__ method.  RTFM. 
http://docs.python.org/tutorial/classes.html#a-first-look-at-classes

Berend van Berkum wrote:
> class MyParser(sgmllib.SGMLParser):
> 
> content = ''
> markup = []
> span_stack = []


--
http://mail.python.org/mailman/listinfo/python-list

Developer needed for Open Source Django/Pinax on-line community

[synarcane]

The Hub is a global community of innovators from every profession,
background and culture working at 'new frontiers' to tackle the
world's
most pressing social, cultural and environmental challenges.  The Hub
has over 1000 members hotdesking in stunning spaces around the world.
The Hub aims to give its members the tools and resources they need to
take their ideas from inception to scale. To this end we are building
an
Open Source online platform to enable Hub members to collaborate
virtually. We have identified Pinax as the platform for building this
over the next few months.

The Hub+ team is the team which provides technology to Hubs globally.
We
are looking for an experienced and enthusiastic Python programmer to
work on a Django/Pinax project in a team committed to Open Source
software and open innovation. If you are interested in inspiring
initiatives and enjoy working in an innovative environment we'd like
you
to join our small team to help build the Hub's ambitious technical
infrastructure.

There will be plenty of opportunities to get involved in other
cutting
edge technical projects with The Hub including real-time comet based
services, network visualisation, collaborative real-time mind-
mapping,
guided navigation, VOIP triggering, and video messaging.

Required
- Experienced Python developer
- Knowledge of Django development
- Javascript
- Basic HTML/CSS
- Based in London (or willing to move)

Preferred*
- Experience in Pinax
- JQuery / YUI
- Active in Open Source communities
- Interest in open innovation
- Twisted
- Comet / Orbited

Please send an e-mail by 23rd February 2009, to synnove.frederi...@the-
hub.net, with some samples of your work, your CV and tell us why you
are interested in joining us.
--
http://mail.python.org/mailman/listinfo/python-list

Re: sgmllib parser keeps old tag data?

[andrew cooke]

Sorry, this reply was delayed (trying to use usenet...) and so now seems
(even more) bad tempered than needed.  Andrew

andrew cooke wrote:
> you are declaring class variables, not instance variables.  you need to
> declare these in an __init__ method.  RTFM.
> http://docs.python.org/tutorial/classes.html#a-first-look-at-classes
> 
> Berend van Berkum wrote:
>> class MyParser(sgmllib.SGMLParser):
>> 
>> content = ''
>> markup = []
>> span_stack = []
> 
> 
> --
> http://mail.python.org/mailman/listinfo/python-list


--
http://mail.python.org/mailman/listinfo/python-list

Python knapsack problem

[Kurioz]

Hi,

I got the assignment to solve the knapsack problem in Python. I have to find 
the solution to put items in a sack (I have only one item A, B and C) which 
maxWeight can't be larger than 6 kilograms.  Solution of this problem should 
be A and C but the only solution I'm getting is B and C which aren't true 
because weight of the B and C is 7 kilograms which is heavier than the 
maxWeight.


If anyone could point me what am I doing wrong I'd be very grateful! Thanks 
in advance!


Code:

#1st array element is weight of an item, and 2nd array element is value
A=[2.0,1.0]
B=[3.0,7.0]
C=[4.0,8.0]
#Checking the values per one kilo
vA=A[1]/A[0]
vB=B[1]/B[0]
vC=C[1]/C[0]
maxWeight=0
print vA,vB,vC
#Checking the values
while maxWeight<6:
   if int(vA)>int(vB) and int(vA)>int(vC):
   maxWeight=maxWeight+A[0]
   vA=0
   print maxWeight

   elif int(vB)>int(vA) and int(vB)>int(vC):
   maxWeight=maxWeight+B[0]
   print maxWeight

   else:
   vC=0
   maxWeight=maxWeight+C[0]
   print maxWeight 


--
http://mail.python.org/mailman/listinfo/python-list

Re: Escaping my own chroot...

[Nick Craig-Wood]

Jean-Paul Calderone  wrote:
>  On Wed, 11 Feb 2009 09:31:56 -0600, Nick Craig-Wood  
> wrote:
> >r0g  wrote:
> >>  I'm writing a linux remastering script in python where I need to chroot
> >>  into a folder, run some system commands and then come out and do some
> >>  tidying up, un-mounting proc & sys etc.
> >>
> >>  I got in there with os.chroot() and I tried using that to get back out
> >>  but that didn't work so... is my script trapped in there forever now or
> >>  is there an un-hacky way to escape?
> >
> >No!
> 
>  If you still have root in the chroot (and you need root to get in there, so
>  it's not implausible that you will), then you can get out.  Googling for
>  "escape chroot" turns up lots of hits.  This page contains a fairly simple,
>  explicit description of how to get out of a chroot:
> 
> http://www.bpfh.net/simes/computing/chroot-break.html
> 
>  See the bulleted list in the "Breaking chroot()" section.  Since you also
>  control the process before the chroot happens, breaking out is even simpler
>  in your case (just open / before you chroot in the first place).  forking
>  before doing the chroot may still be a good idea, but it's not the only
>  solution.

I admit it can be done, but I'm not sure it isn't hacky!

#!/usr/bin/python

"""
Enter a chroot and escape again

Run as root
"""

import os
import sys

def ls(path):
"""List the path"""
print "Directory listing of %r" % path
for f in os.listdir(path):
print ">>", f

def main():
if len(sys.argv) < 2:
print >>sys.stderr, "Need directory to chroot to as an argument"
raise SystemExit(1)
chroot_dir = sys.argv[1]
print "Opening root"
root = os.open("/", os.O_RDONLY)
print "Before chroot"
ls("/")
print "Chrooting to %r" % chroot_dir
os.chroot(chroot_dir)
ls("/")
print "Breaking the chroot"
os.fchdir(root)
for i in range(100):
os.chdir("..")
os.chroot(".")
ls("/")
os.close(root)

if __name__ == "__main__":
main()


I ran this

$ mkdir chroot_test
$ touch chroot_test/in_the_chroot
$ sudo ./chroot_test.py chroot_test

And it produced this

Opening root
Before chroot
Directory listing of '/'
>> lost+found
>> home
>> bin
>> boot
>> proc
>> dev
>> etc
[snip]
Chrooting to 'chroot_test'
Directory listing of '/'
>> in_the_chroot
Breaking the chroot
Directory listing of '/'
>> lost+found
>> home
>> bin
>> boot
>> proc
>> dev
>> etc
[snip]


-- 
Nick Craig-Wood  -- http://www.craig-wood.com/nick
--
http://mail.python.org/mailman/listinfo/python-list

*nix tail -f multiple log files with Thread

[David]

Hi everyone,

I copied a program from C to track multiple log files. I would like to 
be able to print a label when a log file is updated. Here is the program;



#!/usr/bin/python
from threading import Thread
import subprocess
from Queue import Queue

num_threads = 3
queue = Queue()
logfiles = ["/var/log/messages",
"/var/log/apache2/access_log",
"/var/log/apache2/error_log"]

def logtailer(i, q,):
while True:
lfile = q.get()
sudo = "sudo"
tail = "tail"
arg = "-f"
ret = subprocess.call([sudo, tail, arg, lfile])
q.task_done()

for i in range(num_threads):
worker = Thread(target=logtailer, args=(i, queue))
worker.setDaemon(True)
worker.start()

for lfile in logfiles:
queue.put(lfile)

queue.join()


And here is a sample of the output;

[Fri Feb 13 08:58:48 2009] [error] [client 127.0.0.1] File does not 
exist: /var/www/localhost/htdocs/moodle/favicon.ico
[Fri Feb 13 08:58:51 2009] [error] [client 127.0.0.1] File does not 
exist: /var/www/localhost/htdocs/moodle/favicon.ico
Feb 13 09:02:26 opteron sudo:david : TTY=pts/4 ; PWD=/home/david ; 
USER=root ; COMMAND=/bin/tail -f /var/log/apache2/error_log
Feb 13 09:02:26 opteron sudo: pam_unix(sudo:session): session opened for 
user root by david(uid=0)
Feb 13 09:02:26 opteron sudo: pam_unix(sudo:session): session closed for 
user root
Feb 13 09:10:01 opteron cron[10633]: (root) CMD (test -x 
/usr/sbin/run-crons && /usr/sbin/run-crons )

Feb 13 09:18:33 opteron su[10678]: Successful su for root by david
Feb 13 09:18:33 opteron su[10678]: + pts/6 david:root
Feb 13 09:18:33 opteron su[10678]: pam_unix(su:session): session opened 
for user root by david(uid=1000)
127.0.0.1 - - [13/Feb/2009:09:18:47 -0500] "GET 
/theme/custom_corners/favicon.ico HTTP/1.1" 200 894
127.0.0.1 - - [13/Feb/2009:09:18:47 -0500] "GET 
/lib/speller/spellChecker.js HTTP/1.1" 200 15980

127.0.0.1 - - [13/Feb/2009:09:18:47 -0500] "GET / HTTP/1.1" 200 13718
127.0.0.1 - - [13/Feb/2009:09:18:47 -0500] "GET 
/theme/custom_corners/styles.php HTTP/1.1" 200 30709


I would like to be able to add a label like;


[Fri Feb 13 08:58:48 2009] [error] [client 127.0.0.1] File does not 
exist: /var/www/localhost/htdocs/moodle/favicon.ico
[Fri Feb 13 08:58:51 2009] [error] [client 127.0.0.1] File does not 
exist: /var/www/localhost/htdocs/moodle/favicon.ico


Feb 13 09:02:26 opteron sudo:david : TTY=pts/4 ; PWD=/home/david ; 
USER=root ; COMMAND=/bin/tail -f /var/log/apache2/error_log
Feb 13 09:02:26 opteron sudo: pam_unix(sudo:session): session opened for 
user root by david(uid=0)
Feb 13 09:02:26 opteron sudo: pam_unix(sudo:session): session closed for 
user root
Feb 13 09:10:01 opteron cron[10633]: (root) CMD (test -x 
/usr/sbin/run-crons && /usr/sbin/run-crons )

Feb 13 09:18:33 opteron su[10678]: Successful su for root by david
Feb 13 09:18:33 opteron su[10678]: + pts/6 david:root
Feb 13 09:18:33 opteron su[10678]: pam_unix(su:session): session opened 
for user root by david(uid=1000)


127.0.0.1 - - [13/Feb/2009:09:18:47 -0500] "GET 
/theme/custom_corners/favicon.ico HTTP/1.1" 200 894
127.0.0.1 - - [13/Feb/2009:09:18:47 -0500] "GET 
/lib/speller/spellChecker.js HTTP/1.1" 200 15980

127.0.0.1 - - [13/Feb/2009:09:18:47 -0500] "GET / HTTP/1.1" 200 13718
127.0.0.1 - - [13/Feb/2009:09:18:47 -0500] "GET 
/theme/custom_corners/styles.php HTTP/1.1" 200 30709


I can create the label like this;
def label()
print "<%s\n>" % lfile

But I have no idea how to get it to work when a thread is updated.
thanks
-david

--
Powered by Gentoo GNU/LINUX
http://www.linuxcrazy.com
pgp.mit.edu

--
http://mail.python.org/mailman/listinfo/python-list

Re: best set of modules for web automation without javascript

[Paul Rubin]

News123  writes:
> I would just like to retrieve all the field names and default values of
> a form. (Some forms are huge) and wondered thus whether there's already
> a python module parsing a html documents for forms , form fields and
> field vaules, returning an objcet. that could be modified and posted.

BeautifulSoup may be of some help.
--
http://mail.python.org/mailman/listinfo/python-list

Re: best set of modules for web automation without javascript

[News123]

Hi Marco / Paul,

Thanks


I'll look into
mechanize,ClientForm and BeautifulSoup.

All three are now installed. I'll just have to play with them.


bye


N

News123 wrote:
> Hi Joel,
> 
> Thanks,
> This (the urllib2 methods you combined with cookielib) is what I am
> currently trying to do.
> 
> I would just like to retrieve all the field names and default values of
> a form. (Some forms are huge) and wondered thus whether there's already
> a python module parsing a html documents for forms , form fields and
> field vaules, returning an objcet. that could be modified and posted.
> 
> 
> bye
> 
> N
> 
> 
> 
> Joe Riopel wrote:
>> On Fri, Feb 13, 2009 at 9:04 AM, News123  wrote:
>>> Hi,
>>> I'd like to do some web automation with python 2.5
>>> - https:
>>> - a cookiejar
>>> - some forms to be filled in
>>> what is the best set of modules.
>> I have automated some testing of our product, using it's web UI with
>> Python with urllib2 and urrlib. I don't actually fill in the forms, I
>> just recreate the post and set the values of the post variables (so I
>> don't get any form validation).
>>
>> Check out:
>> urllib2.build_opener
>> urllib2.HTTPCookieProcessor
>> urllib2.Request
--
http://mail.python.org/mailman/listinfo/python-list

Fortran array in python (f2py?)...

[tripp]

Hello Folks,

I have a fortran program I use to process several satellite images.  I
currently output the results to a text file (~750 mb) which is then
read by a perl program that outputs a GIS-ready image using GDAL
(www.gdal.org).   There are python libraries for GDAL too.

I'd like to pipe the array directly to python from fortran (instead of
writing it out to a text file).  Is it possible to access an in-memory
fortran array with python?

I've looked at f2py, but I could not tell if this was possible.

Thanks.
-TLowe



--
http://mail.python.org/mailman/listinfo/python-list

Re: best set of modules for web automation without javascript

[Matias Surdi]

You should give a look to Selenium. It's great.

http://www.openqa.org



News123 wrote:

Hi,


I'd like to do some web automation with python 2.5
- https:
- a cookiejar
- some forms to be filled in


what is the best set of modules.

As far as I understood, there is httplib, but it seems (if I understood
well) to be incoompatible with cookielib

I'm a newcomer to webautomation with python and would be thankful for
good suggestions.


I used so far perl with LWP::UserAgent HTTP::Cookies and some module (I
forgot the name) to handle parsing and filling in Forms.

thanks in advance for any pointers opinions


N
--
http://mail.python.org/mailman/listinfo/python-list



--
http://mail.python.org/mailman/listinfo/python-list

Re: something wrong with isinstance

[Terry Reedy]

Gabriel Genellina wrote:
En Fri, 13 Feb 2009 08:43:03 -0200, Peter Otten <__pete...@web.de> 
escribió:



class Type(type):

... def __instancecheck__(self, other): return True
...

class A(metaclass=Type): pass

...

class B: pass

...

isinstance(B(), A)

True

See also

http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubclass 



Ah, ok. Isn't menctioned in the main documentation though.


Lack reported in http://bugs.python.org/issue5250

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python knapsack problem

[Matimus]

On Feb 13, 8:06 am, "Kurioz"  wrote:
> Hi,
>
> I got the assignment to solve the knapsack problem in Python. I have to find
> the solution to put items in a sack (I have only one item A, B and C) which
> maxWeight can't be larger than 6 kilograms.  Solution of this problem should
> be A and C but the only solution I'm getting is B and C which aren't true
> because weight of the B and C is 7 kilograms which is heavier than the
> maxWeight.
>
> If anyone could point me what am I doing wrong I'd be very grateful! Thanks
> in advance!
>
> Code:
>
> #1st array element is weight of an item, and 2nd array element is value
> A=[2.0,1.0]
> B=[3.0,7.0]
> C=[4.0,8.0]
> #Checking the values per one kilo
> vA=A[1]/A[0]
> vB=B[1]/B[0]
> vC=C[1]/C[0]
> maxWeight=0
> print vA,vB,vC
> #Checking the values
> while maxWeight<6:
>     if int(vA)>int(vB) and int(vA)>int(vC):
>         maxWeight=maxWeight+A[0]
>         vA=0
>         print maxWeight
>
>     elif int(vB)>int(vA) and int(vB)>int(vC):
>         maxWeight=maxWeight+B[0]
>         print maxWeight
>
>     else:
>         vC=0
>         maxWeight=maxWeight+C[0]
>         print maxWeight


You will need to check whether each item can fit before adding it.

Currently you are doing:

while there is room in the sac:
add the next most valuable item

You should be doing:

while there is room in the sac:
if the next most valuable item fits
   add it

But... once you fix that you will run into another issue.

You are using ints to compare. Casting floating point values to ints
will always
round down.

vA = 0.5
vB = 2....
vC = 2.0

But..

>>> int(vA)
0
>>> int(vB)
2
>>> int(vC)
2

Matt

--
http://mail.python.org/mailman/listinfo/python-list

Re: Levenshtein word comparison -performance issue

[Terry Reedy]

Gabriel Genellina wrote:
En Fri, 13 Feb 2009 08:16:00 -0200, S.Selvam Siva 
 escribió:



I need some help.
I tried to find top n(eg. 5) similar words for a given word, from a
dictionary of 50,000 words.
I used python-levenshtein module,and sample code is as follow.

def foo(searchword):
disdict={}
for word in self.dictionary-words:
   distance=Levenshtein.ratio(searchword,word)
   disdict[word]=distance
"""
 sort the disdict dictionary by values in descending order
"""
similarwords=sorted(disdict, key=disdict.__getitem__, reverse=True)

return similarwords[:5]


You may replace the last steps (sort + slice top 5) by heapq.nlargest - 
at least you won't waste time sorting 49995 irrelevant words...


There is also no need to build the 5 entry word-distance dictionary.

import heapq, functools

def foo(searchword, n):
  distance = functools.partial(Levenshtein.ratio, searchword)
  return heapq.nlargest(n, words, distance)

If the distances are wanted along with the similar words, I strongly 
suspect that it would be faster to recalculate a small number than to 
generate the dict of 5 pairs.


Anyway you should measure the time taken by the first part 
(Levenshtein), it may be the most demanding. I think there is a C 
extension for this, should be much faster than pure Python calculations.


And such could be dropped into the code above.

Terry Jan Reedy


--
http://mail.python.org/mailman/listinfo/python-list

Re: Fortran array in python (f2py?)...

[MRAB]

tripp wrote:

Hello Folks,

I have a fortran program I use to process several satellite images.  I
currently output the results to a text file (~750 mb) which is then
read by a perl program that outputs a GIS-ready image using GDAL
(www.gdal.org).   There are python libraries for GDAL too.

I'd like to pipe the array directly to python from fortran (instead of
writing it out to a text file).  Is it possible to access an in-memory
fortran array with python?

I've looked at f2py, but I could not tell if this was possible.


How about outputting the results to the Python program via a pipe?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python knapsack problem

[Terry Reedy]

Kurioz wrote:

I got the assignment to solve the knapsack problem in Python. I have to 
find the solution to put items in a sack (I have only one item A, B and 
C) which maxWeight can't be larger than 6 kilograms.  Solution of this 
problem should be A and C but the only solution I'm getting is B and C 
which aren't true because weight of the B and C is 7 kilograms which is 
heavier than the maxWeight.


If anyone could point me what am I doing wrong I'd be very grateful! 
Thanks in advance!


There are several comments I could make, but your immediate problem is 
this: before putting something in the knapsack, you must check whether 
it will exceed the limit.  Since this is a class assignment, I will stop 
there.




Code:

#1st array element is weight of an item, and 2nd array element is value
A=[2.0,1.0]
B=[3.0,7.0]
C=[4.0,8.0]
#Checking the values per one kilo
vA=A[1]/A[0]
vB=B[1]/B[0]
vC=C[1]/C[0]
maxWeight=0
print vA,vB,vC
#Checking the values
while maxWeight<6:
   if int(vA)>int(vB) and int(vA)>int(vC):
   maxWeight=maxWeight+A[0]
   vA=0
   print maxWeight

   elif int(vB)>int(vA) and int(vB)>int(vC):
   maxWeight=maxWeight+B[0]
   print maxWeight

   else:
   vC=0
   maxWeight=maxWeight+C[0]
   print maxWeight
--
http://mail.python.org/mailman/listinfo/python-list



--
http://mail.python.org/mailman/listinfo/python-list

Re: *nix tail -f multiple log files with Thread

[Christian Heimes]

David schrieb:
> Hi everyone,
> 
> I copied a program from C to track multiple log files. I would like to
> be able to print a label when a log file is updated. Here is the program;

Don't use threads for the job. On Unix the preferred way is select()'ing
or poll()'ing multiple file descriptors.

Christian

--
http://mail.python.org/mailman/listinfo/python-list

Re: *nix tail -f multiple log files with Thread

[Joe Riopel]

On Fri, Feb 13, 2009 at 12:03 PM, David  wrote:
> Hi everyone,
>
> I copied a program from C to track multiple log files. I would like to be
> able to print a label when a log file is updated. Here is the program;

Since you're calling tail itself, why not just use tail's ability to
tail multiple files? It too will output the label.

I am using tail from the GNU coreutils, version 5.97.

I did the following, using two xterms:

First, in one xterm window:
$ touch foo bar
$ tail -f foo bar

Then in another xterm:
$ echo "Hi foo" > foo
$ echo "Hi bar" > bar

Back in the first xterm window I see this:
$ tail -f foo bar
==> foo <==

==> bar <==

==> foo <==
Hi foo

==> bar <==
Hi bar
--
http://mail.python.org/mailman/listinfo/python-list

Re: *nix tail -f multiple log files with Thread

[David]

Joe Riopel wrote:

On Fri, Feb 13, 2009 at 12:03 PM, David  wrote:

Hi everyone,

I copied a program from C to track multiple log files. I would like to be
able to print a label when a log file is updated. Here is the program;


Since you're calling tail itself, why not just use tail's ability to
tail multiple files? It too will output the label.

I am using tail from the GNU coreutils, version 5.97.

I did the following, using two xterms:

First, in one xterm window:
$ touch foo bar
$ tail -f foo bar

Then in another xterm:
$ echo "Hi foo" > foo
$ echo "Hi bar" > bar

Back in the first xterm window I see this:
$ tail -f foo bar
==> foo <==

==> bar <==

==> foo <==
Hi foo

==> bar <==
Hi bar


Thanks Joe, sure am glad I asked. Oh well it was good practice as I am 
just learning python, I don't know how I missed that I could tail more 
than one file.


--
Powered by Gentoo GNU/LINUX
http://www.linuxcrazy.com
pgp.mit.edu

--
http://mail.python.org/mailman/listinfo/python-list

problems opening files

[mike]

I have aprogram that manipulates several text files, does some math
and saves new files then exits.

The program worked fine on the original set of data, but with new data
does not open the files as needed.  When I copy and paste the 'open
line' from the module to IDLE the file opens.

section of code that is not working and import statements below:

import statements

# combine point source data

from numpy import *
from string import *
import point_input

section of code-

pathincbp=path+'/sub'+sub+'/cbp_'+npdes+'.txt'
flag='false'
while flag=='false':
try:
cbpin=open(pathincbp,'r')
flag='true'

stopping the module manually--

Traceback (most recent call last):
  File "C:\Python25\point_sources\combine_point_sources_swat.py", line
155, in 
print pathincbp+' not found'
KeyboardInterrupt

IDLE commands---

>>> cbpin=open(pathincbp,'r')
>>> cbpin


I am running in windows xp professional 2002 service pack 3 on a Xeon
dell

Thanks,
Mike


--
http://mail.python.org/mailman/listinfo/python-list

Re: Fortran array in python (f2py?)...

[tripp]

On Feb 13, 1:27 pm, MRAB  wrote:
> tripp wrote:
> > Hello Folks,
>
> > I have a fortran program I use to process several satellite images.  I
> > currently output the results to a text file (~750 mb) which is then
> > read by a perl program that outputs a GIS-ready image using GDAL
> > (www.gdal.org).   There are python libraries for GDAL too.
>
> > I'd like to pipe the array directly to python from fortran (instead of
> > writing it out to a text file).  Is it possible to access an in-memory
> > fortran array with python?
>
> > I've looked at f2py, but I could not tell if this was possible.
>
> How about outputting the results to the Python program via a pipe?

The array I'd like to output is 8500 x 7500.  Would that be possible?
I'm NOT RAM limited.
--
http://mail.python.org/mailman/listinfo/python-list

Re: problems opening files

[MRAB]

mike wrote:

I have aprogram that manipulates several text files, does some math
and saves new files then exits.

The program worked fine on the original set of data, but with new data
does not open the files as needed.  When I copy and paste the 'open
line' from the module to IDLE the file opens.

section of code that is not working and import statements below:

import statements

# combine point source data

from numpy import *
from string import *
import point_input

section of code-

pathincbp=path+'/sub'+sub+'/cbp_'+npdes+'.txt'
flag='false'
while flag=='false':
try:
cbpin=open(pathincbp,'r')
flag='true'

stopping the module manually--

Traceback (most recent call last):
  File "C:\Python25\point_sources\combine_point_sources_swat.py", line
155, in 
print pathincbp+' not found'
KeyboardInterrupt

IDLE commands---


cbpin=open(pathincbp,'r')
cbpin



I am running in windows xp professional 2002 service pack 3 on a Xeon
dell


What exception is the open() raising? From the partial code you've
provided I can't tell whether the 'try' is catching any exception. The
traceback just tells me that you interrupted the script, not what "is
not working" actually means.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Untangling pythonWin and IDLE Processes on XP Pro

[Scott David Daniels]

W. eWatson wrote:

Terry Reedy wrote:

W. eWatson wrote:

 From Diez above.
What does *NOT* work is writing a Tkinter-based app in idle, and to 
run it

*FROM INSIDE* idle. Instead, open your explorer and double-click on the
pyhton-file your app is in. That's all that there is to it.

So this is the absolute truth? No wiggle room? One can never use a 
Tkinter program with IDLE, and execute it successfully. So IDLE 
doesn't issue a standard warning that says, "Get out of here with 
your Tkinter program, it will fail when you try to run it here. You 
have entered Tkinter hell. Good-bye."


Re-read my post about kids fighting to control a television.  Maybe 
they work together, maybe they crash the TV.  Hard to predict.


***ANY*** Python program that tries to grab and control the same 
resources that TK does may conflict with it.  There is no way that 
IDLE can have a list of, for instance, all event-grabbing mainloop 
programs.


OK, enough tinkering with the code and others matters on my end trying 
to find a work around. Somehow after much successful use of IDLE's 
execution facility, I've stepped on an invisible banana peel. I think 
it's evident that I'm not going around this problem easily with the IDLE 
execution attempts, and that another solution is required.

That's correct, but you still don't understand _why_ it is correct.
I suggest you re-read the thread and try to understand everything you
are being told.

First, I think somewhere up the thread someone suggested that Active 
pythonWin is not dependent upon Tk, correct? Therefore, it is immune 
from such problems, correct?


Wrong.  I was the one who said that ActiveState had a product to debug
Python programs across a nertwork connection.  The product is _not_
ActivePython (the freely distributed system), but rather the Komodo IDE,
which does cost money.

Finally, we can probably agree that I can continue to use IDLE for 
editing and syntax checking, but to "guarantee" successful execution of 
the program, I can just double-click on the py file in my folder. 
Perhaps there is a better way than clicking on it in the folder. For 
example, putting it on the desktop. As I look at the folder, previous 
copies only differ by a digit, I can easily find myself executing an 
earlier version, differing as Dev4, to Dev5 at the end of each name.


OK, you are using the oldest and least useful revision control system,
"rename and remember."  I'd suggest you get and use bazaar, but you'll
just ask for shortcuts on how to use it without understanding what it does.

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Upgrading standard library module

[Bryan]

I have a Python v2.5.2 server running and I found some undesirable
behavior in the xmlrpclib module that is included with that version of
Python.  The xmlrpclib version that is included with Python 2.6
changes the behavior for the better.  I nervous about upgrading my
Python install to 2.6 on this server because I remember reading in the
docs of a library I use that it targets the 2.5 branch.  What is the
best way to only upgrade the xmlrpclib in my 2.5 install?

Also, have you all had problems with libraries not specifying which
Python version they target?  I can't find a requirements for the
formencode library.  It is kind of scary upgrading my Python server
install without being sure all my libraries will work.  Experiences??

Bryan
--
http://mail.python.org/mailman/listinfo/python-list

pdb in 3.0 very buggy (Win XP Home)

[Aaron Brady]

Hi, got a freeze when running 'pdb' in 3.0.

The program executes correctly with the command 'c', but freezes part
way through when running successive 'n' commands.  Platform Windows XP
Home.  Python 3.0 (r30:67507, Dec  3 2008, 20:14:27) [MSC v.1500 32
bit (Intel)] on win32.  The 'list' command was also not working
properly, printing '[EOF]' many places, even when prompted with
arguments for 'first' and 'last'.

/Screen dump:

C:\Documents and Settings\usr\Desktop\working>\programs
\python30\python -m pdb picktest.py

--Return--
> (5)A()->None
(Pdb) c
0
1
2
The program finished and will be restarted
--Return--
> (5)A()->None
(Pdb) n
--Return--
> c:\programs\python30\lib\io.py(757)closed()->False
-> return self.raw.closed
(Pdb) n
--Return--
> c:\programs\python30\lib\io.py(1467)closed()->False
-> return self.buffer.closed
(Pdb) n
--Return--
> c:\programs\python30\lib\encodings\cp437.py(19)encode()->b'0'
-> return codecs.charmap_encode(input,self.errors,encoding_map)[0]
(Pdb) n
--Return--
> c:\programs\python30\lib\io.py(757)closed()->False
-> return self.raw.closed
(Pdb) n
0--Return--
> c:\programs\python30\lib\io.py(1060)write()->1
-> return written
(Pdb) n
--Return--
> c:\programs\python30\lib\io.py(1498)write()->1
-> return length
(Pdb) n
--Return--
> c:\programs\python30\lib\io.py(757)closed()->False
-> return self.raw.closed
(Pdb) n
--Return--
> c:\programs\python30\lib\io.py(1467)closed()->False
-> return self.buffer.closed
(Pdb) n
--Return--
> c:\programs\python30\lib\encodings\cp437.py(19)encode()->b'\n'
-> return codecs.charmap_encode(input,self.errors,encoding_map)[0]
(Pdb) n
--Return--
> c:\programs\python30\lib\io.py(757)closed()->False
-> return self.raw.closed
(Pdb) n

--Return--
> c:\programs\python30\lib\io.py(1060)write()->1
-> return written
(Pdb) n

/Program:

class A:
def f( self ):
self.x= 0
print( self.x )
def g( self ):
self.x+= 1
print( self.x )

a= A()
a.f()
a.g()
a.g()

--
http://mail.python.org/mailman/listinfo/python-list

Re: Upgrading standard library module

[Mike Driscoll]

On Feb 13, 2:42 pm, Bryan  wrote:
> I have a Python v2.5.2 server running and I found some undesirable
> behavior in the xmlrpclib module that is included with that version of
> Python.  The xmlrpclib version that is included with Python 2.6
> changes the behavior for the better.  I nervous about upgrading my
> Python install to 2.6 on this server because I remember reading in the
> docs of a library I use that it targets the 2.5 branch.  What is the
> best way to only upgrade the xmlrpclib in my 2.5 install?
>
> Also, have you all had problems with libraries not specifying which
> Python version they target?  I can't find a requirements for the
> formencode library.  It is kind of scary upgrading my Python server
> install without being sure all my libraries will work.  Experiences??
>
> Bryan

Well, you could put your current install in a virtualenv and then try
the new version of Python 2.6 in another virtualenv. See
http://pypi.python.org/pypi/virtualenv for more info.

As long as your 3rd party packages are pure python, you should be ok.
If they depend on c/c++ headers, then you may have issues.

Mike
--
http://mail.python.org/mailman/listinfo/python-list

Re: Fortran array in python (f2py?)...

[Robert Kern]

On 2009-02-13 11:59, tripp wrote:

Hello Folks,

I have a fortran program I use to process several satellite images.  I
currently output the results to a text file (~750 mb) which is then
read by a perl program that outputs a GIS-ready image using GDAL
(www.gdal.org).   There are python libraries for GDAL too.

I'd like to pipe the array directly to python from fortran (instead of
writing it out to a text file).  Is it possible to access an in-memory
fortran array with python?

I've looked at f2py, but I could not tell if this was possible.


f2py makes Python extension modules that wrap FORTRAN code. Basically, what you 
would have to do is use f2py to wrap the subroutines of your FORTRAN program; 
your Python program will be the main driver. It is usually recommended that your 
Python program allocate the memory in the form of a numpy array, and pass it 
into your FORTRAN subroutines to be filled. F77 fixed arrays are well-supported; 
F90 allocatable arrays and pointers are not. You may need a fixed array shim 
layer around your Fortran 90 code if it uses these features.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: Upgrading standard library module

[Jason Scheirer]

On Feb 13, 12:42 pm, Bryan  wrote:
> I have a Python v2.5.2 server running and I found some undesirable
> behavior in the xmlrpclib module that is included with that version of
> Python.  The xmlrpclib version that is included with Python 2.6
> changes the behavior for the better.  I nervous about upgrading my
> Python install to 2.6 on this server because I remember reading in the
> docs of a library I use that it targets the 2.5 branch.  What is the
> best way to only upgrade the xmlrpclib in my 2.5 install?
>
> Also, have you all had problems with libraries not specifying which
> Python version they target?  I can't find a requirements for the
> formencode library.  It is kind of scary upgrading my Python server
> install without being sure all my libraries will work.  Experiences??
>
> Bryan

Python has always been pretty backwards-compatible and has a nice
roadmap for upgrades (with new languages features living in __future__
for a version or so before they're in the main version). With the
exception of 2.X->3, you can usually assume that any pure-Python
modules written for 2.x will work in 2.(x+1), and assuming they don't
issue any warnings, also in 2.(x+2).

What behavior, exactly, do you not like in xmlrpclib? Diffing the 2.5
and 2.6, only significant differences I see are checks for True/False
as builtins left over from pre-2.4 and some datetime handling.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Upgrading standard library module

[Bryan]

On Feb 13, 1:52 pm, Jason Scheirer  wrote:
> On Feb 13, 12:42 pm, Bryan  wrote:
>
> > I have a Python v2.5.2 server running and I found some undesirable
> > behavior in the xmlrpclib module that is included with that version of
> > Python.  The xmlrpclib version that is included with Python 2.6
> > changes the behavior for the better.  I nervous about upgrading my
> > Python install to 2.6 on this server because I remember reading in the
> > docs of a library I use that it targets the 2.5 branch.  What is the
> > best way to only upgrade the xmlrpclib in my 2.5 install?
>
> > Also, have you all had problems with libraries not specifying which
> > Python version they target?  I can't find a requirements for the
> > formencode library.  It is kind of scary upgrading my Python server
> > install without being sure all my libraries will work.  Experiences??
>
> > Bryan
>
> Python has always been pretty backwards-compatible and has a nice
> roadmap for upgrades (with new languages features living in __future__
> for a version or so before they're in the main version). With the
> exception of 2.X->3, you can usually assume that any pure-Python
> modules written for 2.x will work in 2.(x+1), and assuming they don't
> issue any warnings, also in 2.(x+2).
>
> What behavior, exactly, do you not like in xmlrpclib? Diffing the 2.5
> and 2.6, only significant differences I see are checks for True/False
> as builtins left over from pre-2.4 and some datetime handling.

The xmlrpclib in my 2.5.2 install does not allow the marshaling of my
custom objects.  It checks the type of my object, and if it isn't in a
hard-coded list of types, then a "Cannot marshal object of 
type" exception message shows up.  The version in my Windows 2.6
install has a fall back case where if the type is not in the hard-
coded list, it simply calls Object.__dict__ to get a graph of the
object's values that can be marshaled into xmlrpc.

I am using xmlrpc through Pylons.  As a workaround, instead of
returning my custom objects in my xmlrpc server functions, I return
MyObject.__dict__

I also did not see anything in the revision log in the source file
about this change.
--
http://mail.python.org/mailman/listinfo/python-list

Re: *nix tail -f multiple log files with Thread

[Jarkko Torppa]

On 2009-02-13, Christian Heimes  wrote:
> David schrieb:
>> Hi everyone,
>> 
>> I copied a program from C to track multiple log files. I would like to
>> be able to print a label when a log file is updated. Here is the program;
>
> Don't use threads for the job. On Unix the preferred way is select()'ing
> or poll()'ing multiple file descriptors.

 File descriptors associated with regular files always select
 true  for  ready  to  read, ready to write, and error condi-
 tions.

 Regular files always poll TRUE for reading and writing.

You have to either sleep or use kevent/completion/...

-- 
Jarkko Torppa, Elisa
--
http://mail.python.org/mailman/listinfo/python-list

Re: Untangling pythonWin and IDLE Processes on XP Pro

[Rhodri James]

On Fri, 13 Feb 2009 11:13:38 -, W. eWatson   
wrote:


OK, enough tinkering with the code and others matters on my end trying  
to find a work around. Somehow after much successful use of IDLE's  
execution facility, I've stepped on an invisible banana peel. I think  
it's evident that I'm not going around this problem easily with the IDLE  
execution attempts, and that another solution is required.


Congratulations, we've only been telling you this for the last few days.
I wonder, is there any chance that you've noticed the solution given?

First, I think somewhere up the thread someone suggested that Active  
pythonWin is not dependent upon Tk, correct?


Someone certainly suggested that something is based on Microsoft
Foundation Classes, which isn't very likely to be Tk-based :-)
Whatever that something is, I'm pretty sure it isn't called "Active
pythonWin".


Therefore, it is immune from such problems, correct?


No.

Let me put it like this.  Your Tkinter program is listening out for
events like windows being moved, the mouse being clicked, keys being
pressed and so on.  IDLE also listens out for a selection of events
of the same sort.  A different graphical IDE will do the same, but will
trap and interpret the events in an entirely different manner that is
probably not even a little bit compatible with your program.  When
you run your program from inside *any* IDE, not just IDLE, it's a bit
of a lottery as to whether your program gets an event, or the IDE
does.  Chances are, *both* need to see it, at which point you're
stuck.

It is possible to do this successfully, but only in a very limited
way and only if you're very careful.  If you think either of those
conditions hold, you are wrong.


Second, maybe I missed it above, but when I posted the output from the  
program that showed the failure, was there anything that said, "IDLE  
problem" or would even give a clue that's the culprit?


How can it?  It's not IDLE's problem, it's yours.

Finally, we can probably agree that I can continue to use IDLE for  
editing and syntax checking, but to "guarantee" successful execution of  
the program, I can just double-click on the py file in my folder.  
Perhaps there is a better way than clicking on it in the folder.


Typing at a command prompt.


For example, putting it on the desktop.


This causes an extra file read as Windows indirects through the desktop
link.  It's unlikely to be a noticeable delay at startup, but I'd
hesitate to call it "better".

As I look at the folder, previous copies only differ by a digit, I can  
easily find myself executing an earlier version, differing as Dev4, to  
Dev5 at the end of each name.


I'd suggest spending a while reading up on version control systems.

--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: A little bit else I would like to discuss

[Simon Hibbs]

On 13 Feb, 02:53, azrael  wrote:

> All I hear when I talk to people who own or eork in SW companies is
> "Python? Isn't that that small scripting language. This will never
> bring a real application." I am tired of hearing this. Even Microsoft
> implemented Python. Only because the GUI building I am thinking about
> of moving to IronPython. But this great comunity holds me bac on
> CPython.

I don't agree at all that Python can't compete with Visual Studio for
GUI development. There are Python modules for Eclipse if you need an
enterprise class IDE. For GUI design the PyQT GUI toolkit includes QT
Designer - a fully featured graphical GUI builder every bit as capable
as the one in Visual Studio. In fact I much prefer working in QT
Designer for GUI layout, the Signals/Slots model for hooking up to the
application  logic is just so much more elegant and robust.

If Eclipse is a bit heavyweight for you, and you're after a Pyhton
equivalent to VB, I can't recommend Eric enough. It's a Python IDE
written in Python using the PyQT toolkit, and integrates directly with
QTDesigner. IMHO it's far superior to anything Microsoft has to offer,
with full native support for MacOS X, Linux and Windows.

The only fly in the ointment in licensing. The QT toolkit will be
fully GPL and LGPL from version 4.4, but PyQT itself has a dual GPL/
commercial licensing structure. Still, it's relatively cheap and well
worth it if you're goingt to do commercial development.

Simon Hibbs
--
http://mail.python.org/mailman/listinfo/python-list

Turning a signature-changing decorator into a signature-preserving one

[Gustavo Narea]

Hello, everybody.

I have this signature-changing decorator

which I
want to turn into a signature-preserving one. Here's my try
,
but I get
this error:
http://paste.chrisarndt.de/paste/33e06be6e6d74e49b05c45534dfcc9fe?wrap=no

What am I doing wrong? I think it looks like this example:
http://pypi.python.org/pypi/decorator#async

Thanks in advance.

  - Gustavo.

PS: functols.wrap is not an option because my code has to work with
Python 2.4+.
--
http://mail.python.org/mailman/listinfo/python-list

Re: A little bit else I would like to discuss

[Basilisk96]

On Feb 12, 10:39 pm, Damon  wrote:
> * Like R, every time there is a new version of Python, the repository
> should rebuild the packages, for all supported platforms, and make
> available all those that compile cleanly. R also forces you to write
> properly structured documentation for every exposed function, before
> the repository will accept it.

A very good idea indeed. I would love to start using Py3k today, but I
am still on 2.5 because I depend on win32 binaries for the majority of
my library packages. I can build some myself, but not all. A
repository of prebuilt binaries that stay in step with currently
available language releases would be most welcome.

Just my $0.02,
-Basilisk96
--
http://mail.python.org/mailman/listinfo/python-list

Which to install on my windows vista laptop?

[Sue]

Hi,

Which python 2.6.1 file should I download for my windows vista home
premium laptop?  (32bit, AMD turion 64 x2)

the windows x86 MSI installer or the windows AMD64 MSI installer? 

Thank you!

Sue

--
http://mail.python.org/mailman/listinfo/python-list

Re: how can this iterator be optimized?

[Basilisk96]

On Feb 12, 1:15 am, Steven D'Aprano
 wrote:
> > I usually strive
> > for comprehensions if a for loop can be reduced to such.
>
> Any particular reason?

Only two.
1.) I was impressed by their clarity and conciseness when I first
discovered them.
2.) I also read now and then that simple list comprehensions are
faster when compared with their for-loop equivalents because of the
way comprehensions are implemented under the hood. My example is a far
cry from a "simple" comprehension, however. :)


> If there's only one call to func(), and you ignore the (probably) fixed
> cost of jumping into a generator each time, then it shouldn't make any
> difference.
>
> If you are comparing one call to func() in a for loop versus three calls
> to func() in a list comp or generator expression, then of course the for
> loop will be more efficient.

I agree. I would rather call func() only once per iteration in any
case. I will revise it to a plain for loop with a single call.

Thanks,
-Basilisk96
--
http://mail.python.org/mailman/listinfo/python-list

Re: Which to install on my windows vista laptop?

[Kurioz]

If you are using 32bit OS(which you are) then use the x86 MSI installer. 
AMD64 MSI installer is used only when you are using 64bit OS.
"Sue"  wrote in message 
news:mailman.9515.1234572128.3487.python-l...@python.org...

Hi,

Which python 2.6.1 file should I download for my windows vista home
premium laptop?  (32bit, AMD turion 64 x2)

the windows x86 MSI installer or the windows AMD64 MSI installer?

Thank you!

Sue


--
http://mail.python.org/mailman/listinfo/python-list

Re: Levenshtein word comparison -performance issue

[Basilisk96]

On Feb 13, 5:42 am, "Gabriel Genellina" 
wrote:
> You may replace the last steps (sort + slice top 5) by heapq.nlargest - at  
> least you won't waste time sorting 49995 irrelevant words...
> Anyway you should measure the time taken by the first part (Levenshtein),  
> it may be the most demanding. I think there is a C extension for this,  
> should be much faster than pure Python calculations.

subdist: http://pypi.python.org/pypi/subdist/0.2.1
It uses a modified "fuzzy" version of the Levenshtein algorithm, which
I found more useful than the strict version. The only quirk to it is
that it accepts nothing but unicode. Other than that, it's a keeper.
It is extremely fast.

Cheers,
-Basilisk96
--
http://mail.python.org/mailman/listinfo/python-list

Re: A little bit else I would like to discuss

[Benjamin Kaplan]

On Fri, Feb 13, 2009 at 7:22 PM, Basilisk96  wrote:

> On Feb 12, 10:39 pm, Damon  wrote:
> > * Like R, every time there is a new version of Python, the repository
> > should rebuild the packages, for all supported platforms, and make
> > available all those that compile cleanly. R also forces you to write
> > properly structured documentation for every exposed function, before
> > the repository will accept it.
>
> A very good idea indeed. I would love to start using Py3k today, but I
> am still on 2.5 because I depend on win32 binaries for the majority of
> my library packages. I can build some myself, but not all. A
> repository of prebuilt binaries that stay in step with currently
> available language releases would be most welcome.
>
> Just my $0.02,
> -Basilisk96


With Py3K, it's a little bit more complicated than rebuilding all the
packages. Because it broke compatibility, all of the C extensions have to be
rewritten before they'll work. That's why it's taking longer to create
Python 3 packages than to create 2.6 versions.

>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Easier to wrap C or C++ libraries?

[argo785]

When creating a Python binding to a C or C++ library, which is easier
to wrap, the C lib or the C++ one? Given a choice, if you had to
choose between using one of two libs, one written in C, the other in C+
+ -- both having approximately the same functionality -- which would
you rather deal with from your Python code?

It would seem to me that there's fewer design considerations when
wrapping a library written in C; you just wrap the functions. However,
since Python supports OOP nicely, it might also be that wrapping C++
code *could* also be staightforward... Are there many pitfalls when
having to map C++'s notion of OO to Python?
--
http://mail.python.org/mailman/listinfo/python-list

ANNOUNCING allmydata.org "Tahoe", the Least-Authority Filesystem, v1.3

[zooko]

Folks:

This "Cloud Storage" system is written entirely in Python except for  
the CPU-intensive parts (cryptography and erasure coding), which are  
provided as Python extension modules.  Thanks for making Python such  
a high-quality and effective tool!


Regards,

Zooko

ANNOUNCING allmydata.org "Tahoe", the Least-Authority Filesystem, v1.3

We are pleased to announce the release of version 1.3.0 of "Tahoe", the
Least Authority Filesystem.

Tahoe-LAFS is a secure, decentralized, fault-tolerant filesystem.  All
of the source code is available under a choice of two Free Software,
Open Source licences.

This filesystem is encrypted and distributed over multiple peers in
such a way it continues to function even when some of the peers are
unavailable, malfunctioning, or malicious.

Here is the one-page explanation of the security and fault-tolerance
properties that it offers:

http://allmydata.org/source/tahoe/trunk/docs/about.html

This is the successor to v1.2, which was released July 21, 2008 [1].
This is a major new release, adding a repairer, an efficient backup
command, support for large files, an (S)FTP server, and much more.

See the NEWS file [2] and the known_issues.txt file [3] for more
information.

In addition to the many new features of Tahoe itself, a crop of related
projects have sprung up, including Tahoe frontends for Windows and
Macintosh, two front-ends written in JavaScript, a Tahoe plugin for
duplicity, a Tahoe plugin for TiddlyWiki, a project to create a new
backup tool, CIFS/SMB integration, an iPhone app, and three incomplete
Tahoe frontends for FUSE. See Related Projects on the wiki: [4].


COMPATIBILITY

The version 1 branch of Tahoe is the basis of the consumer backup
product from Allmydata, Inc. -- http://allmydata.com .

Tahoe v1.3 is fully compatible with the version 1 branch of Tahoe.
Files written by v1.3 clients can be read by clients of all versions
back to v1.0 unless the file is too large -- files greater than about
12 GiB (depending on the configuration) can't be read by older clients.
v1.3 clients can read files produced by clients of all versions since
v1.0.  v1.3 servers can serve clients of all versions back to v1.0 and
v1.3 clients can use servers of all versions back to v1.0 (but can't
upload large files to them).

This is the fourth release in the version 1 series.  We believe that
this version of Tahoe is stable enough to rely on as a permanent store
of valuable data.  The version 1 branch of Tahoe will be actively
supported and maintained for the forseeable future, and future versions
of Tahoe will retain the ability to read files and directories produced
by Tahoe v1 for the forseeable future.


WHAT IS IT GOOD FOR?

With Tahoe, you can distribute your filesystem across a set of
computers, such that if some of the computers fail or turn out to be
malicious, the entire filesystem continues to be available, thanks to
the remaining computers.  You can also share your files with other
users, using a simple and flexible access control scheme.

Because this software is new, we do not categorically recommend it as
the sole repository of data which is extremely confidential or
precious.  However, we believe that erasure coding, strong encryption,
Free/Open Source Software and careful engineering make Tahoe safer than
common alternatives, such as RAID, removable drive, tape, or "on-line
storage" or "Cloud storage" systems.

This software comes with extensive unit tests [5], and there are no
known security flaws which would compromise confidentiality or data
integrity.  (For all currently known issues please see the
known_issues.txt file [2].)

This release of Tahoe is suitable for the "friendnet" use case [6] --
it is easy to create a filesystem spread over the computers of you and
your friends so that you can share disk space and files.


LICENCE

You may use this package under the GNU General Public License, version
2 or, at your option, any later version.  See the file "COPYING.GPL"
[7] for the terms of the GNU General Public License, version 2.

You may use this package under the Transitive Grace Period Public
Licence, version 1.0.  The Transitive Grace Period Public Licence has
requirements similar to the GPL except that it allows you to wait for
up to twelve months after you redistribute a derived work before
releasing the source code of your derived work. See the file
"COPYING.TGPPL.html" [8] for the terms of the Transitive Grace Period
Public Licence, version 1.0.

(You may choose to use this package under the terms of either licence,
at your option.)


INSTALLATION

Tahoe works on Linux, Mac OS X, Windows, Cygwin, and Solaris, and
probably most other systems.  Start with "docs/install.html" [9].


HACKING AND COMMUNITY

Please join us on the mailing list [10].  Patches that extend and
improve Tahoe are gratefully accepted -- the RoadMap page [11] shows
the next improvements that we plan to make and CREDITS [12] lists the
names of people who've contributed to th

RELEASED Python 3.0.1

[Barry Warsaw]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On behalf of the Python development team, I'm happy to announce the  
availability of Python 3.0.1, the first bug fix release of Python  
3.0.  Version 3.0.1 fixes dozens of bugs reported since the release of  
Python 3.0 on December 3rd, 2008.


Python 3.0 represents a major milestone in Python's history.  This new  
version of the language is incompatible with the 2.x line of releases,  
while remaining true to BDFL Guido van Rossum's vision.


For more information, links to documentation, and downloadable  
distributions, see the Python 3.0.1 release page:


http://www.python.org/download/releases/3.0.1/

To report bugs in Python 3.0.1, please submit them to the issue  
tracker at:


http://bugs.python.org/

Enjoy!
Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSZYpSnEjvBPtnXfVAQLQwgP/WSHp12dJVpEYtEOL/X8ynCQACriij9AM
PgT6SacbMJLbsy84CTGA1lxF4NdEUQMY1IYz0do/aZ0+nBkSoy7SlkOVcncysLSC
hVyTVlWQBdh63yA8QUk1I5dMbKeNpbCqRRgvSHaBrVdVz9mDM/r/L+j9lhBW4Cam
2lHLjRdQaG0=
=vy0O
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easier to wrap C or C++ libraries?

[Daniel Fetchinson]

> When creating a Python binding to a C or C++ library, which is easier
> to wrap, the C lib or the C++ one? Given a choice, if you had to
> choose between using one of two libs, one written in C, the other in C+
> + -- both having approximately the same functionality -- which would
> you rather deal with from your Python code?
>
> It would seem to me that there's fewer design considerations when
> wrapping a library written in C; you just wrap the functions. However,
> since Python supports OOP nicely, it might also be that wrapping C++
> code *could* also be staightforward... Are there many pitfalls when
> having to map C++'s notion of OO to Python?

There is no question about it in my mind that wrapping C is easier.
Reason being that python is written in C and not C++.

Cheers,
Daniel

-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown
--
http://mail.python.org/mailman/listinfo/python-list

Re: pdb in 3.0 very buggy (Win XP Home)

[Benjamin Peterson]

Aaron Brady  gmail.com> writes:

> 
> Hi, got a freeze when running 'pdb' in 3.0.

This is a known issue because of the rewrite of the IO library in 3.0. It will
hopefully be fixed in 3.1. See http://bugs.python.org/issue3618 for more
information. 




--
http://mail.python.org/mailman/listinfo/python-list

Re: Easier to wrap C or C++ libraries?

[Chris Rebert]

On Fri, Feb 13, 2009 at 5:35 PM,   wrote:
> When creating a Python binding to a C or C++ library, which is easier
> to wrap, the C lib or the C++ one? Given a choice, if you had to
> choose between using one of two libs, one written in C, the other in C+
> + -- both having approximately the same functionality -- which would
> you rather deal with from your Python code?
>
> It would seem to me that there's fewer design considerations when
> wrapping a library written in C; you just wrap the functions. However,
> since Python supports OOP nicely, it might also be that wrapping C++
> code *could* also be staightforward... Are there many pitfalls when
> having to map C++'s notion of OO to Python?

You're asking two separate questions here.

(1) For Python bindings, is it easier to wrap C, or C++ libraries?
As Daniel points out, C is easier to wrap because CPython itself is
written in C, and C++ cannot be linked directly to C.

(2) Is it easier to use C or C++ bindings from Python code?
Interface-wise, the C++ bindings would be easier to work with at the
Python level as they are object-oriented.

In practice, C libraries are usually wrapped because that's easier to
do. However, to compensate for their procedural interface, people
often write another module on top of the raw Python bindings to expose
a public interface that appears more object-oriented, and make the
lower-level module internal/private.

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

String concatenation performance with +=

[Sammo]

String concatenation has been optimized since 2.3, so using += should
be fairly fast.

In my first test, I tried concatentating a 4096 byte string 1000 times
in the following code, and the result was indeed very fast (12.352 ms
on my machine).

import time
t = time.time()
mydata = ""
moredata = "A"*4096
for i in range(1000):
mydata += moredata # 12.352 ms
print "%0.3f ms"%(1000*(time.time() - t))

However, I got a different result in my second test, which is
implemented in a class with a feed() method. This test took 4653.522
ms on my machine, which is 350x slower than the previous test!

class StringConcatTest:
def __init__(self):
self.mydata = ""

def feed(self, moredata):
self.mydata += moredata # 4653.522 ms

test = StringConcatTest()
t = time.time()
for i in range(1000):
test.feed(moredata)
print "%0.3f ms"%(1000*(time.time() - t))

Note that I need to do something to mydata INSIDE the loop, so please
don't tell me to append moredata to a list and then use "".join after
the loop.

Why is the second test so much slower?
--
http://mail.python.org/mailman/listinfo/python-list

Re: String concatenation performance with +=

[Benjamin Peterson]

Sammo  gmail.com> writes:

> 
> String concatenation has been optimized since 2.3, so using += should
> be fairly fast.

This is implementation dependent and shouldn't be relied upon.

> 
> Note that I need to do something to mydata INSIDE the loop, so please
> don't tell me to append moredata to a list and then use "".join after
> the loop.

Then why not just mutate the list and then call "".join?

> 
> Why is the second test so much slower?

Probably several reasons:

1. Function call overhead is quite large compared to these simple operations.
2. You are resolving attribute names.




--
http://mail.python.org/mailman/listinfo/python-list

Re: A little bit else I would like to discuss

[Steven D'Aprano]

Martin wrote:

> Hi,
> 
> at first I wanted to file this under meta-discussions, but your lost
> paragraph got me thinking...
> 
> 2009/2/12 Christian Heimes :
>> Nobody is going to stop you from creating a large bundle of useful
>> extensions as long as you follow the licenses. In fact lots of people
>> may appreciate a bundle. But the Python core package will always stay
>> small and agile.
> 
> How does "small and agile" work with "batteries included"?

Well, it certainly isn't agile without batteries.


>>From my point of view:
> 
> agile::
>  Would describe faster extension of the standard lib (rrd, yaml should
> IMHO already be in the standard lib). 

No, agile describes the language. It's easy to get things done *with*
Python. You're asking whether the Python-dev development process of getting
packages added to the standard library is agile. It absolutely is not, and
that is by design: python-dev doesn't want to fill the std lib up with
unnecessary, unused and buggy batteries just because one or two people
request them.

> I'm pretty sure other people 
> want to see other modules, but that's what agile + standard lib would
> mean for me. (Also I'm sometimes confused by the naming of modules but
> that's a different story)

The process for getting modules added to the std lib is on the Python
website.

http://www.python.org/dev/peps/pep-0002/

If you have a library you want added, go right ahead and make a PEP.


> small::
>  just the opposite of "batteries included"

Says who? Batteries can be small, and adding a few new batteries to the pile
doesn't necessarily make the pile significantly bigger.

Small means it doesn't have unnecessary batteries.


 
> My absolute favorite would be
> 
>  * python as just python (no standard lib)

Would be a toy.

>  * a (rather) fast moving standard lib available as an addon download

No no no, the std lib needs to be conservative, so that if you write code
that relies on the std lib, you can be confident that it will work
anywhere. The core language and std library are conservative, slowly
changing, so that developers have stability. You don't have to write code
like this:

if version == 2.5:
code block
elif version == 2.5.1b:
different code block
elif version == 2.5.1:
still different code block
elif version == 2.5.2:
if exists(module):
code block
else:
something different again
elif version == 2.5.3:
 ...


If you want to live on the bleeding edge, with rapidly changing APIs and
libraries, they have a repository for that. It's called "the Internet".




-- 
Steven

--
http://mail.python.org/mailman/listinfo/python-list

Re: RELEASED Python 3.0.1

[Benjamin Kaplan]

On Fri, Feb 13, 2009 at 9:15 PM, Barry Warsaw  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On behalf of the Python development team, I'm happy to announce the
> availability of Python 3.0.1, the first bug fix release of Python 3.0.
>  Version 3.0.1 fixes dozens of bugs reported since the release of Python 3.0
> on December 3rd, 2008.
>
> Python 3.0 represents a major milestone in Python's history.  This new
> version of the language is incompatible with the 2.x line of releases, while
> remaining true to BDFL Guido van Rossum's vision.
>
> For more information, links to documentation, and downloadable
> distributions, see the Python 3.0.1 release page:
>
>http://www.python.org/download/releases/3.0.1/
>
> To report bugs in Python 3.0.1, please submit them to the issue tracker at:
>
>http://bugs.python.org/
>
> Enjoy!
> Barry



Any chance of getting a Mac installer for this one?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Untangling pythonWin and IDLE Processes on XP Pro

[W. eWatson]

Scott David Daniels wrote:

W. eWatson wrote:

Terry Reedy wrote:

W. eWatson wrote:

 From Diez above.
What does *NOT* work is writing a Tkinter-based app in idle, and to 
run it

*FROM INSIDE* idle. Instead, open your explorer and double-click on the
pyhton-file your app is in. That's all that there is to it.

So this is the absolute truth? No wiggle room? One can never use a 
Tkinter program with IDLE, and execute it successfully. So IDLE 
doesn't issue a standard warning that says, "Get out of here with 
your Tkinter program, it will fail when you try to run it here. You 
have entered Tkinter hell. Good-bye."


Re-read my post about kids fighting to control a television.  Maybe 
they work together, maybe they crash the TV.  Hard to predict.


***ANY*** Python program that tries to grab and control the same 
resources that TK does may conflict with it.  There is no way that 
IDLE can have a list of, for instance, all event-grabbing mainloop 
programs.


OK, enough tinkering with the code and others matters on my end trying 
to find a work around. Somehow after much successful use of IDLE's 
execution facility, I've stepped on an invisible banana peel. I think 
it's evident that I'm not going around this problem easily with the 
IDLE execution attempts, and that another solution is required.

That's correct, but you still don't understand _why_ it is correct.
I suggest you re-read the thread and try to understand everything you
are being told.

First, I think somewhere up the thread someone suggested that Active 
pythonWin is not dependent upon Tk, correct? Therefore, it is immune 
from such problems, correct?


Wrong.  I was the one who said that ActiveState had a product to debug
Python programs across a nertwork connection.  The product is _not_
ActivePython (the freely distributed system), but rather the Komodo IDE,
which does cost money.
I'm pretty sure it wasn't you, and had no relationship to what you brought 
up earlier several messages up the thread. There are other forums.


Finally, we can probably agree that I can continue to use IDLE for 
editing and syntax checking, but to "guarantee" successful execution 
of the program, I can just double-click on the py file in my folder. 
Perhaps there is a better way than clicking on it in the folder. For 
example, putting it on the desktop. As I look at the folder, previous 
copies only differ by a digit, I can easily find myself executing an 
earlier version, differing as Dev4, to Dev5 at the end of each name.


OK, you are using the oldest and least useful revision control system,
"rename and remember."  I'd suggest you get and use bazaar, but you'll
just ask for shortcuts on how to use it without understanding what it does.
It works for me, and, frankly, I'm not interested in going to Linux, SunOS 
or other "revision systmes". These are way in my distant past, and the only 
reason I'm currently, and begrudgingly, once again writing programs is that 
the Python software program I am using is limited in its ability. I've 
finally, after 2-3 years of hoping someone else would do it, taken up the 
torch to add new features. Frankly, I'd rather be doing something else with 
my time.


And, yes, you are somewhat correct in your earlier assessment of my goals, 
the sooner this is over the better. You may not like my philosophy, but it 
serves me well at the moment, and I'm moving ahead nicely now.


As I recall from the old movie Desk Set, a conversation between their two 
characters regarding a puzzle he was about to give her as a test of her 
office abilities: Tracy cautions Hepburn, "Never assume!" before relating 
the famous "detective" problem. Never assume.


Nevertheless, thank you for your responses.
Be kind to your keyboard.

Cheers.


--Scott David Daniels
scott.dani...@acm.org



--
   W. eWatson

 (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
  Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet

Web Page: 

--
http://mail.python.org/mailman/listinfo/python-list

Re: Untangling pythonWin and IDLE Processes on XP Pro

[W. eWatson]

...

How can it?  It's not IDLE's problem, it's yours.

Finally, we can probably agree that I can continue to use IDLE for 
editing and syntax checking, but to "guarantee" successful execution 
of the program, I can just double-click on the py file in my folder. 
Perhaps there is a better way than clicking on it in the folder.


Typing at a command prompt.


For example, putting it on the desktop.


This causes an extra file read as Windows indirects through the desktop
link.  It's unlikely to be a noticeable delay at startup, but I'd
hesitate to call it "better".

As I look at the folder, previous copies only differ by a digit, I can 
easily find myself executing an earlier version, differing as Dev4, to 
Dev5 at the end of each name.


I'd suggest spending a while reading up on version control systems.


See my response to Scott. Thanks for your reply.

--
   W. eWatson

 (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
  Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet

Web Page: 

--
http://mail.python.org/mailman/listinfo/python-list

Re: Embarrasing questio

[Steven D'Aprano]

TechieInsights wrote:

> On Feb 12, 9:03 am, Catherine Heathcote
>  wrote:
>> But I just cant find it. How do I do an or, as in c/c++'s ||? Just
>> trying to do something simple, the python equivilent of:
>>
>> if(i % 3 == 0 || i % 5 == 0)
>>
>> Thanks.
> 
> in 2.5 and above you can do
> if any(i%3 == 0, i%5 == 0)

[nitpick: you need an additional set of brackets inside the call to any]

I love it! A totally unnecessary creation of a tuple and a function call
instead of a straight-forward 'or' expression. Can we make this simple task
even more obfuscated?

not all( (i%3 != 0, i%5 !=0) )

Still not convoluted enough. 

all( (sum(int(c) for c in str(i))%3 in (1, 2), 
str(i).endswith(chr(53)) - 1, 
str(i).endswith(chr(48)) - 1) ) - 1

Hmmm... how can I get rid of that pesky %3?

all( (divmod( sum(int(c) for c in str(i)), 
41^42)[1] in (1, 2), 
str(i).endswith(chr(53)) - 1, 
str(i).endswith(chr(48)) - 1) ) - 1

Okay, that makes my brain hurt. I think it will do.

:-)



-- 
Steven

--
http://mail.python.org/mailman/listinfo/python-list

Re: Turning a signature-changing decorator into a signature-preserving one

[Michele Simionato]

On Feb 14, 12:56 am, Gustavo Narea  wrote:
> Hello, everybody.
>
> I have this signature-changing decorator
> 
> which I
> want to turn into a signature-preserving one. Here's my try
> ,
> but I get
> this 
> error:http://paste.chrisarndt.de/paste/33e06be6e6d74e49b05c45534dfcc9fe?wra...
>
> What am I doing wrong? I think it looks like this 
> example:http://pypi.python.org/pypi/decorator#async
>
> Thanks in advance.
>
>   - Gustavo.
>
> PS: functols.wrap is not an option because my code has to work with
> Python 2.4+.

Difficult to see what is happening, since you have so many
dependencies. However, from the traceback I would say that
you are passing to the decorator a function with a __name__
attribute which is None. I would insert a call to pdb here

def __call__(self, func):
import pdb; pdb.set_trace()
return decorator(self.call, func)

and I would inspect func.__name__.

HTH,

 M.Simionato
--
http://mail.python.org/mailman/listinfo/python-list