Re: How to test if one dict is subset of another?

[Paul Rubin]

"Jay Tee" <[EMAIL PROTECTED]> writes:
> for j in jobs:
>if (j.get('user') == 'jeff' and j.get('state')=='running') :
>   do_something()

Sounds like you need some backing data structures, like indexes
in a database, e.g. (untested, uses the cool new defaultdicts of 2.5):

   index = defaultdict(lambda: defaultdict(set))
   for j in jobs:
 for k in j's dict:
index[k][j.get(k)].add(j)

Now for

>if j.subset_attr({'user' : 'jeff', 'state' : 'running'}) :
>   do_something()

you'd just write:

for j in (index['user']['jeff'] & index['state']['running']):
do_something()
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Checking for EOF in stream

[Nathan]

On 2/20/07, Nathan <[EMAIL PROTECTED]> wrote:
> On 2/19/07, Gabriel Genellina <[EMAIL PROTECTED]> wrote:
> > En Mon, 19 Feb 2007 21:50:11 -0300, GiBo <[EMAIL PROTECTED]> escribió:
> >
> > > Grant Edwards wrote:
> > >> On 2007-02-19, GiBo <[EMAIL PROTECTED]> wrote:
> > >>>
> > >>> Classic situation - I have to process an input stream of unknown length
> > >>> until a I reach its end (EOF, End Of File). How do I check for EOF? The
> > >>> input stream can be anything from opened file through sys.stdin to a
> > >>> network socket. And it's binary and potentially huge (gigabytes), thus
> > >>> "for line in stream.readlines()" isn't really a way to go.
> > >>>
> > >>> For now I have roughly:
> > >>>
> > >>> stream = sys.stdin
> > >>> while True:
> > >>> data = stream.read(1024)
> > >> if len(data) == 0:
> > >>  break  #EOF
> > >>> process_data(data)
> > >
> > > Right, not a big difference though. Isn't there a cleaner / more
> > > intuitive way? Like using some wrapper objects around the streams or
> > > something?
> >
> > Read the documentation... For a true file object:
> > read([size]) ... An empty string is returned when EOF is encountered
> > immediately.
> > All the other "file-like" objects (like StringIO, socket.makefile, etc)
> > maintain this behavior.
> > So this is the way to check for EOF. If you don't like how it was spelled,
> > try this:
> >
> >if data=="": break
> >
> > If your data is made of lines of text, you can use the file as its own
> > iterator, yielding lines:
> >
> > for line in stream:
> >  process_line(line)
> >
> > --
> > Gabriel Genellina
> >
> > --
> > http://mail.python.org/mailman/listinfo/python-list
> >
>
> Not to beat a dead horse, but I often do this:
>
> data = f.read(bufsize):
> while data:
> # ... process data.
> data = f.read(bufsize)
>
>
> -The only annoying bit it the duplicated line.  I find I often follow
> this pattern, and I realize python doesn't plan to have any sort of
> do-while construct, but even still I prefer this idiom.  What's the
> concensus here?
>
> What about creating a standard binary-file iterator:
>
> def blocks_of(infile, bufsize = 1024):
> data = infile.read(bufsize)
> if data:
> yield data
>
>
> -the use would look like this:
>
> for block in blocks_of(myfile, bufsize = 2**16):
> process_data(block) # len(block) <= bufsize...
>


(ahem), make that iterator something that works, like:

def blocks_of(infile, bufsize = 1024):
data = infile.read(bufsize)
while data:
yield data
data = infile.read(bufsize)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Checking for EOF in stream

[Nathan]

On 2/19/07, Gabriel Genellina <[EMAIL PROTECTED]> wrote:
> En Mon, 19 Feb 2007 21:50:11 -0300, GiBo <[EMAIL PROTECTED]> escribió:
>
> > Grant Edwards wrote:
> >> On 2007-02-19, GiBo <[EMAIL PROTECTED]> wrote:
> >>>
> >>> Classic situation - I have to process an input stream of unknown length
> >>> until a I reach its end (EOF, End Of File). How do I check for EOF? The
> >>> input stream can be anything from opened file through sys.stdin to a
> >>> network socket. And it's binary and potentially huge (gigabytes), thus
> >>> "for line in stream.readlines()" isn't really a way to go.
> >>>
> >>> For now I have roughly:
> >>>
> >>> stream = sys.stdin
> >>> while True:
> >>> data = stream.read(1024)
> >> if len(data) == 0:
> >>  break  #EOF
> >>> process_data(data)
> >
> > Right, not a big difference though. Isn't there a cleaner / more
> > intuitive way? Like using some wrapper objects around the streams or
> > something?
>
> Read the documentation... For a true file object:
> read([size]) ... An empty string is returned when EOF is encountered
> immediately.
> All the other "file-like" objects (like StringIO, socket.makefile, etc)
> maintain this behavior.
> So this is the way to check for EOF. If you don't like how it was spelled,
> try this:
>
>if data=="": break
>
> If your data is made of lines of text, you can use the file as its own
> iterator, yielding lines:
>
> for line in stream:
>  process_line(line)
>
> --
> Gabriel Genellina
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

Not to beat a dead horse, but I often do this:

data = f.read(bufsize):
while data:
# ... process data.
data = f.read(bufsize)


-The only annoying bit it the duplicated line.  I find I often follow
this pattern, and I realize python doesn't plan to have any sort of
do-while construct, but even still I prefer this idiom.  What's the
concensus here?

What about creating a standard binary-file iterator:

def blocks_of(infile, bufsize = 1024):
data = infile.read(bufsize)
if data:
yield data


-the use would look like this:

for block in blocks_of(myfile, bufsize = 2**16):
process_data(block) # len(block) <= bufsize...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: builtin set literal

[zefciu]

Paul Rubin wrote:
> There's even a sentiment in some pythonistas to get rid of the [] and {}
> notations for lists and dicts, using list((1,2,3)) and dict((1,2),(3,4))
> for [1,2,3] and {1:2, 3:4} respectively.

Wow.  This makes Python twice more LISPy, than <1, 2, 3> and {-} make it
C-ish and Perlish.

Frop Python-zen:
Simple is better than complex.
Flat is better than nested.

Sets are good.  Sets are fun.  Sets are pythonish (the programmer codes
the logical side of a data structure, bothering less 'bout the
technical).  I think, they deserve their literal notation.

zefciu
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bypassing __setattr__ for changing special attributes

[[EMAIL PROTECTED]]

On 20 fév, 05:39, "George Sakkis" <[EMAIL PROTECTED]> wrote:
> I was kinda surprised that setting __class__ or __dict__ goes through
> the __setattr__ mechanism, like a normal attribute:

But __class__ and __dict___ *are* 'normal' attributes...


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help Required for Choosing Programming Language

[Laurent Pointal]

Mark Morss a écrit :
> On Feb 16, 4:22 pm, [EMAIL PROTECTED] wrote:
>> I am VB6 programmer and wants to start new programming language but i
>> am unable to deciced.
>>
>> i have read about Python, Ruby and Visual C++. but i want to go
>> through with GUI based programming language like VB.net
>>
>> so will you please guide me which GUI based language has worth with
>> complete OOPS Characteristics
>>
>> will wait for the answer
>>
>> hope to have a right direction from you Programmer
> 
> Good grief.  I suppose it is Microsoft to whom we owe the idea that
> there could be such a thing as a "GUI based" programming language.

Maybe HyperCard (and its language, HyperTalk) come before VB (SuperCard
and MetaCard too)... and its Apple...

Whatever we - as professional developers - think about these
tools/languages - its very nice to allow not-developers people to build
the tools they need, even if these are not perfect in a conceptual view,
these are software with users' need filled.

Note: I agree, in professionnal world with software service during long
years, with software growing to manage more and more tasks, some
languages and tools are better than others.

Different needs, different personal experience, different skills...
different languages.


L.Pointal.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Forking SocketServer daemon -- updating state

[Nick Craig-Wood]

Reid Priedhorsky <[EMAIL PROTECTED]> wrote:
>  I am implementing a forking SocketServer daemon that maintains significant
>  internal state (a graph that takes ~30s to build by fetching from a SQL
>  database, and eventually further state that may take up to an hour to
>  build).
> 
>  I would like to be able to notify the daemon that it needs to update its
>  state. Because it forks for each new request, a request handler can't
>  update the state because then only the child would have the new state.
> 
>  One idea I had was to use signals. Is it safe to write a signal handler
>  that does extensive work (several seconds)? Seems like even so, it might
>  be tricky to do this without race conditions.

In general no it isn't safe.

However due to the way python handles its interrupts (it sets a flag
in its own signal handler which the bytecode interpreter examines and
acts upon when it is safe to do so) you won't corrupt python if you do
this.

I'd still recommend the set a flag and do it in your mainloop approach
though if you can.

>  Another possibility is that the signal handler simply sets a
>  needs_update flag, which I could check for in a handle_request()
>  loop. The disadvantage here is that the update wouldn't happen
>  until after the next request is handled, and I would like the state
>  to be available for that next request.  A solution might be to send
>  a signal followed by a dummy request, which seems a bit awkward.

Can't you get SocketServer to timeout and return you control
regularly?

-- 
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: f---ing typechecking

[Nick Craig-Wood]

Neil Cerutti <[EMAIL PROTECTED]> wrote:
>  On 2007-02-14, Farshid Lashkari <[EMAIL PROTECTED]> wrote:
> > Szabolcs Nagy wrote:
> > L=[1]
> > L.extend((1,))
> > L
> >> [1, 1]
> >
> > Are list.extend() and list concatenation supposed to behave
> > differently? I always thought concatenation was just shorthand
> > for calling extend().
> 
>  They are different. list.extend() mutates the list, returning
>  None, while the + operator returns a new, concatenated list.
> 
>  += on the other hand works very similarly to list.extend().

It does make an inconsistency though...

>>> L=[1]
>>> L+=(1,)
>>> L
[1, 1]

Wheras

>>> L=[1]
>>> L=L+(1,)
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: can only concatenate list (not "tuple") to list
>>> 

Ie

x += a

does not equal

x = x + a

which it really should for all types of x and a

(That is the kind of statement about which I'm sure someone will post
a perfectly reasonable counterexample ;-)

-- 
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
-- 
http://mail.python.org/mailman/listinfo/python-list

Can I reverse eng a .pyc back to .py?

[Steven W. Orr]

The short story is that someone left, but before he left he checked in a 
.pyc and then both the directory was destroyed and the backups all got 
shredded (don't ask*). Is there anything that can be extracted? I looked 
on the web and the subject seems to get different answers, all old.

Any joy?

TIA


* Murphy's Ultimate Corollary: If it could have gone wrong earlier and it 
didn't, it ultimately would have better for it to have.

-- 
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: WHAT IS THIS?

[James Stroud]

Marc 'BlackJack' Rintsch wrote:
> In <[EMAIL PROTECTED]>, James Stroud wrote:
> 
>> Better would be to remove windows xp and get another operating system.
> 
> Yeah XP is sooo ooold, the OP should install Vista.  Or did you mean a
> real OS instead of just another one?  ;-)
> 
> SCNR,
>   Marc 'BlackJack' Rintsch

I meant a real OS.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Can I reverse eng a .pyc back to .py?

[Felipe Almeida Lessa]

On 2/19/07, Steven W. Orr <[EMAIL PROTECTED]> wrote:
> The short story is that someone left, but before he left he checked in a
> .pyc and then both the directory was destroyed and the backups all got
> shredded (don't ask*). Is there anything that can be extracted? I looked
> on the web and the subject seems to get different answers, all old.

Only for .pyc's of Python versions upto 2.3:
http://packages.debian.org/unstable/python/decompyle

-- 
Felipe.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python-list Digest, Vol 41, Issue 286

[KoDer]

> Subject: Can I reverse eng a .pyc back to .py?
> The short story is that someone left, but before he left he checked in a
> .pyc and then both the directory was destroyed and the backups all got
> TIA

https://svn.cs.pomona.edu/sys/src/konane/pyunparse.py

Need add new AST nodes if You want to use it with python2.5.
-- 
K.Danilov aka KoDer
-- 
http://mail.python.org/mailman/listinfo/python-list

wxPython: non-GUI thread launching new frame? Delegates?

[cyberco]

In my wxPython app a non-GUI thread (that reads info from the network)
tries to open a frame to show the new info. This results in my app
hanging (which is not too surprising). Coming from a C# environment I
wonder if there is some sort of delegate mechanism in wxPython to do
this sort of thing.

2B

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Django, one more newbie question

[Piet van Oostrum]

> Boris Ozegovic <[EMAIL PROTECTED]> (BO) wrote:

>BO> Umm, can somebody tell me which language is this one:
>BO> {% if latest_poll_list %}
>BO> 
>BO> {% for poll in latest_poll_list %}
>BO> {{ poll.question }}
>BO> {% endfor %}
>BO> 
>BO> {% else %}
>BO> No polls are available.
>BO> {% endif %}

>BO> Whole tutorial is on this page:
>BO> http://www.djangoproject.com/documentation/tutorial3/
 
>BO> endfor, endif, {{?  :)

It is the Django template language.
-- 
Piet van Oostrum <[EMAIL PROTECTED]>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: [EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxPython: non-GUI thread launching new frame? Delegates?

[Diez B. Roggisch]

cyberco wrote:

> In my wxPython app a non-GUI thread (that reads info from the network)
> tries to open a frame to show the new info. This results in my app
> hanging (which is not too surprising). Coming from a C# environment I
> wonder if there is some sort of delegate mechanism in wxPython to do
> this sort of thing.

Not sure how wx deals with this, but one thing you might explore is the
possibility to add a timer in the GUI-thread, that polls a thread-filled
queue.

Other toolkits as Qt have means to insert an extra event in the event queue
of the gui-thread in a thread-agnostic way, maybe wx has that too.

Googling...
...
...
...

... finished

http://mail.python.org/pipermail/python-list/2005-August/335467.html

"""
You need another way to pass completion information between the downloader
thread and the main thread; the simplest way is to define a custom wx
Event, and wxPostEvent from the downloader thread when it completes (
and when the gauge should be updated).  wxPostEvent is safe to call from
non-eventloop threads.
The main thread's wx event loop just spins, properly updating all other
parts of the GUI, and receiving events from the downloader thread.

ANother approach is to have a thread-safe Queue and have the main
thread/event loop
poll the queue with queue.get_nowait() periodically (typically 0.1-1 sec).
The downloader thread shares the queue object and puts data structures
(typically
class instances, strings, or ints) that indicate status updates.
"""

So - both options a viable. And read to the end, the twisted-approach
certainly is the most clean one.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Weird result returned from adding floats depending on order I add them

[joanne matthews (RRes-Roth)]

I'm getting different results when I add up a list of floats depending
on the order that I list the floats. For example, the following returns
False:
def check():
totalProp=0
inputs=[0.2,0.2,0.2,0.1,0.2,0,0.1]
for each in inputs:

totalProp+=each
print "totalProp=",totalProp
if totalProp != 1:
print "Your proportions must add up to 1"
   
return False
return True

However, if I swap, the 4th and 5th list items like this:

totalProp=0
inputs=[0.2,0.2,0.2,0.2,0.1,0,0.1]
for each in inputs:

   totalProp+=each
   print "totalProp=",totalProp
   if totalProp != 1:
print "Your proportions must add up to 1"
   
return False
   return True

I get True returned. Can anyone tell me whats going on and how I can
avoid the problem. Thanks

Joanne Matthews
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird result returned from adding floats depending on order I add them

[Laurent Pointal]

joanne matthews (RRes-Roth) a écrit :
> I'm getting different results when I add up a list of floats depending
> on the order that I list the floats. For example, the following returns
> False:
> def check():
>   totalProp=0
>   inputs=[0.2,0.2,0.2,0.1,0.2,0,0.1]
>   for each in inputs:
> 
>   totalProp+=each
>   print "totalProp=",totalProp
>   if totalProp != 1:
>   print "Your proportions must add up to 1"
>
>   return False
>   return True
> 
> However, if I swap, the 4th and 5th list items like this:
> 
> totalProp=0
> inputs=[0.2,0.2,0.2,0.2,0.1,0,0.1]
> for each in inputs:
> 
>totalProp+=each
>print "totalProp=",totalProp
>if totalProp != 1:
> print "Your proportions must add up to 1"
>
> return False
>return True
> 
> I get True returned. Can anyone tell me whats going 

Its related to the internal representation of real numbers using a
finite number of binary digits - intermediate additions may (here the
order is have an impact) produce results which have no representation,
and lead to dismiss of an epsilon value.

http://en.wikipedia.org/wiki/Floating_point

> on and how I can
> avoid the problem. Thanks

Use an ad-hoc library with numerical types using a different
representation (other posters may give examples of libraries they use).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bypassing __setattr__ for changing special attributes

[Steven D'Aprano]

On Mon, 19 Feb 2007 23:18:02 -0800, Ziga Seilnacht wrote:

> George Sakkis wrote:
>> I was kinda surprised that setting __class__ or __dict__ goes through
>> the __setattr__ mechanism, like a normal attribute:
>>
>> class Foo(object):
>> def __setattr__(self, attr, value):
>> pass
>>
>> class Bar(object):
>> pass
>>
>> >>> f = Foo()
>> >>> f.__class__ = Bar
>> >>> print f.__class__ is Foo
>> True
>>
>> Is there a way (even hackish) to bypass this, or at least achieve
>> somehow the same goal (change f's class) ?
>>
>> George
> 
 object.__setattr__(f, '__class__', Bar)
 f.__class__ is Bar
> True


This version is arguably more "correct", although a tad longer to write,
and doesn't need you to hard-code the class superclass:

super(f.__class__, f).__setattr__('__class__', Bar)


But what surprised me was that this *didn't* work:

>>> f = Foo()
>>> f.__dict__['__class__'] = Bar
>>> f.__class__



Unless I'm confused, it looks like instance.__class__ bypasses the usual
lookup mechanism (instance.__dict__, then instance.__class__.__dict__) for
some reason.

>>> Foo.x = 1  # stored in class __dict__
>>> f.x
1
>>> f.__dict__['x'] = 2  # stored in instance __dict__
>>> f.x
2
>>> Foo.x
1

But __class__ doesn't behave like this. Why?



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird result returned from adding floats depending on order I add them

[John McCallum]

Hi,

> I'm getting different results when I add up a list of floats depending
> on the order that I list the floats. For example, the following returns
>
> [snip summation]
>
> if totalProp != 1:

>From a numerical analysis point of view, never ever do this. The values
you are adding are approximations to the numbers you require, you then
test for equality (the real no no)...

There is even a section in the Python tutorial about it:

http://docs.python.org/tut/node16.html

Cheers,
John McCallum
Edinburgh
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pep 3105: the end of print?

[Steven D'Aprano]

On Tue, 20 Feb 2007 00:44:24 +, Peter mayne wrote:

> Steven D'Aprano wrote:
>> If Python 3 dropped the print
>> statement and replaced it with official_print_function(), how would that
>> help you in your goal to have a single code base that will run on both
>> Python 2.3 and Python 3, while still using print?
> 
> Is there any reason why official_print_function isn't sys.stdout.write?

Why would you want a convenience function like print to take one import,
three look-ups and 16 characters instead of always available, one look-up
and five characters?

> I can't remember the last time I used print in actual code (apart from 
> short-lived debugging lines), so I'm bewildered as to why print seems to 
> be so important.

print is important for the convenience, for short-lived debugging, and for
use in the interactive interpreter.


-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird result returned from adding floats depending on order I add them

[John Machin]

On Feb 20, 11:29 pm, "joanne matthews (RRes-Roth)"
<[EMAIL PROTECTED]> wrote:
> I'm getting different results when I add up a list of floats depending
> on the order that I list the floats.

This is quite expected. Floating point arithmetic is subject to
rounding errors.

[doesn't add to 1.0]
> inputs=[0.2,0.2,0.2,0.1,0.2,0,0.1]
>
> However, if I swap, the 4th and 5th list items like this:
>
> inputs=[0.2,0.2,0.2,0.2,0.1,0,0.1]
[adds to 1.0]

What is happening:

print repr(whatever_you_are_puzzled_by) is a Very Good Idea (TM).

>>> a = [0.2, 0.2, 0.2, 0.1, 0.2, 0.1]
>>> b = [0.2, 0.2, 0.2, 0.2, 0.1, 0.1]
>>> sum(a)
1.0002
>>> sum(b)
1.0
>>> tot = 0.0
>>> for x in a:
...tot += x
...print repr(x), repr(tot)
...
0.20001 0.20001
0.20001 0.40002
0.20001 0.60009
0.10001 0.70007
0.20001 0.90013
0.10001 1.0002
>>> tot = 0.0
>>> for x in b:
...tot += x
...print repr(x), repr(tot)
...
0.20001 0.20001
0.20001 0.40002
0.20001 0.60009
0.20001 0.80004
0.10001 0.90002
0.10001 1.0
>>>

As you can see, 0.1 and 0.2 can't be represented exactly as floating
point numbers. Consequently there is only a rough chance that they
will add up to what you think they should add up to.

Fixes:

(1) Round the sums to a suitable precision.
>>> round(sum(a), 3)
1.0
>>> round(sum(b), 3)
1.0
>>>

(2) Test against a range, rather than for equality:

>>> abs(sum(a) - 1.0) < 0.001
True
>>> abs(sum(b) - 1.0) < 0.001
True
>>>

(3) Use the Decimal module

(4) Google this group (or the Python cookbok, I forget which) for
fancy algorithms for doing accurate sums of lists of floats.

HTH,
John

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: file io (lagged values) newbie question

[Steven D'Aprano]

On Mon, 19 Feb 2007 22:17:42 -0800, hiro wrote:

> Hey there, I'm currently doing data preprocessing (generating lagged
> values for a time series) and I'm having some difficulties trying to
> write a file to disk.  A friend of mine, wrote this quick example for
> me:

If that's a quick example (well over 100 lines), I'd hate to see your
idea of a long example.

Can you cut out all the irrelevant cruft and just show:

(1) the actual error you are getting
(2) the SMALLEST amount of code that demonstrates the error

Try to isolate if the problem is in *writing* the file or *generating*
the time series.

Hint: instead of one great big lump of code doing everything, write AT
LEAST two functions: one to read values from a file and generate a time
series, and a second to write it to a file.

That exercise will probably help you discover what the problem is, and
even if it doesn't, you'll have narrowed it down from one great big lump
of code to a small function.

To get you started, here's my idea for the second function:
(warning: untested)

def write_series(data, f):
"""Write a time series data to file f. 

data should be a list of integers.
f should be an already opened file-like object.
"""
# Convert data into a string for writing.
s = str(data)
s = s[1:-1]  # strip the leading and trailing [] delimiters
s = s.replace(',', '') # delete the commas
# Now write it to the file object
f.write(s)
f.write('\n')


And this is how you would use it:

f = file('myfile.txt', 'w')
# get some time series data somehow...
data = [1, 2, 3, 4, 5]  # or something else
write_series(data, f)
# get some more data
data = [2, 3, 4, 5, 6]
write_series(data, f)
# and now we're done
f.close()


Hope this helps.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

CherryPy/Turbogears on server not controlled by me

[Brian Blais]

Hello,

I was wondering if there is a way to run CherryPy/Turbogears on a server that I 
don't
have root access to.  If I just choose a random port, I think the security guys 
on
the server would get annoyed at me.  What are my options?  I can talk to the 
admin,
but they are very slow/reluctant to make changes...it took me a couple months 
to get
them to upgrade to 2.4 from 2.3 last year, even when 2.5 was out.


thanks,

Brian Blais

-- 
-

  [EMAIL PROTECTED]
  http://web.bryant.edu/~bblais

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: file io (lagged values) newbie question

[Jussi Salmela]

hiro kirjoitti:
> Hey there, I'm currently doing data preprocessing (generating lagged
> values for a time series) and I'm having some difficulties trying to
> write a file to disk.  A friend of mine, wrote this quick example for
> me:
>

> 
> tweaked code:
> ---
> f=open('c:/kaka.txt','r')
> array=f.readlines()
> f.close()
> f=open('c:/kakaDump.txt','w')
> lineSize = 4
> skip = 4
> condition = 1
> startIndex = 0
> 
> for letter in array:
>line = []
>startIndex = array.index(letter)
> 
>for indexNum in range(startIndex, startIndex + (skip - 1), 1):
>if indexNum > (len(array) - 1):
>break
>else:
>line.append(array[indexNum])
> 
>for indexNum in range(startIndex + skip, (startIndex +
> lineSize) + 1, 1):
>if indexNum > (len(array) - 1):
>break
>else:
>line.append(array[indexNum])
> 
>f.writelines(line)
> 
> ---
> C:\>more kakaDump.txt
> 1
> 2
> 3
> 5
> 2
> 3
> 4
> 6
> 3
> 4
> 5
> 74
> 5
> 6
> 5
> 6
> 76
> 77
> 
> For those familiar with neural networks, the input file is a time
> series and the output file needs to have 3 lagged variables for
> training and a (two time steps ahead) variable for the target.  Ie:
> 
> input file
> 1
> 2
> 3
> 4
> 5
> 6
> 7
> 
> output file
> 1 2 3 5
> 2 3 4 6
> 3 4 5 7
> 4 5 6
> 5 6 7
> 6 7
> 7
> 
> Thanks in advanced,
> 
> 
> D.
> 

I think your file kaka.txt lacks a CR-LF i.e. '\n' i.e. "Enter" after 
the last line.

To get the desired output format you also need to drop the CR-LF:s after 
each line to have the required values printed on the same line. Here's 
my version of your code with a couple remarks added:

#-
f = open('kaka.txt','r')
# The Windows root directory C:\ is a special directory
# designed to be used by Windows itself. To put it
# politely: it's unwise to do program development in
# that directory
array = f.readlines()
f.close()
# This drops the '\n' from each line:
array = [x[:-1] for x in array]
#print array
f = open('kakaDump.txt','w')
lineSize = 4
skip = 4
condition = 1
startIndex = 0

for letter in array:
line = []
startIndex = array.index(letter)

for indexNum in range(startIndex, startIndex + (skip - 1), 1):
if indexNum > (len(array) - 1):
break
else:
line.append(array[indexNum])
# This adds a space between each item in a row
# and after the last item, but it's "only" a space:
line.append(' ')

for indexNum in range(startIndex + skip, (startIndex +
lineSize) + 1, 1):
if indexNum > (len(array) - 1):
break
else:
line.append(array[indexNum])
# This completes the line:
line.append('\n')

f.writelines(line)
f.close()
#-

I also have my own completely different version which to me looks 
cleaner than yours but as they say: "Beauty is in the eye of the beholder"

#-
lineSize = 4
lsm1 = lineSize - 1

f = open('kaka.txt','r')
inData = f.read()
f.close()

inLst = inData.split()
inCount = len(inLst)
inLst += [' ']*lineSize
print inLst

f = open('kakaDump.txt','w')
for ind,elem in enumerate(inLst):
 if ind == inCount: break
 for i in range(lsm1): f.write('%s ' % inLst[ind + i])
 f.write('%s\n' % inLst[ind + lineSize])
f.close()
#-

HTH,
Jussi
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CherryPy/Turbogears on server not controlled by me

[Diez B. Roggisch]

Brian Blais wrote:

> Hello,
> 
> I was wondering if there is a way to run CherryPy/Turbogears on a server
> that I don't
> have root access to.  If I just choose a random port, I think the security
> guys on
> the server would get annoyed at me.  

Why should they? Opening anything networking will open a port. And if you
have shell access, you can point PYTHONPATH to a directory of your choice
and let easy_install install the packages required there.

I don't see a problem. And _if_ they get mad at you, well - there isn't
anything you can do about that I presume apart from telling them to shut
up - because without a port, there is no webapp...

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Sorting directory contents

[Wolfgang Draxinger]

H folks,

I got, hmm not really a problem, more a question of elegance:

In a current project I have to read in some files in a given
directory in chronological order, so that I can concatenate the
contents in those files into a new one (it's XML and I have to
concatenate some subelements, about 4 levels below the root
element). It all works, but somehow I got the feeling, that my
solution is not as elegant as it could be:

src_file_paths = dict()
for fname in os.listdir(sourcedir):
fpath = sourcedir+os.sep+fname
if not match_fname_pattern(fname): continue
src_file_paths[os.stat(fpath).st_mtime] = fpath
for ftime in src_file_paths.keys().sort():
read_and_concatenate(src_file_paths[ftime])

of course listdir and sorting could be done in a separate
function, but I wonder if there was a more elegant approach.

Wolfgang Draxinger
-- 
E-Mail address works, Jabber: [EMAIL PROTECTED], ICQ: 134682867

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bypassing __setattr__ for changing special attributes

[George Sakkis]

On Feb 20, 7:57 am, Steven D'Aprano
<[EMAIL PROTECTED]> wrote:
> On Mon, 19 Feb 2007 23:18:02 -0800, Ziga Seilnacht wrote:
> > George Sakkis wrote:
> >> I was kinda surprised that setting __class__ or __dict__ goes through
> >> the __setattr__ mechanism, like a normal attribute:
>
> >> class Foo(object):
> >> def __setattr__(self, attr, value):
> >> pass
>
> >> class Bar(object):
> >> pass
>
> >> >>> f = Foo()
> >> >>> f.__class__ = Bar
> >> >>> print f.__class__ is Foo
> >> True
>
> >> Is there a way (even hackish) to bypass this, or at least achieve
> >> somehow the same goal (change f's class) ?
>
> >> George
>
>  object.__setattr__(f, '__class__', Bar)
>  f.__class__ is Bar
> > True
>
> This version is arguably more "correct", although a tad longer to write,
> and doesn't need you to hard-code the class superclass:
>
> super(f.__class__, f).__setattr__('__class__', Bar)
>
> But what surprised me was that this *didn't* work:
>
> >>> f = Foo()
> >>> f.__dict__['__class__'] = Bar
> >>> f.__class__
>
> 
>
> Unless I'm confused, it looks like instance.__class__ bypasses the usual
> lookup mechanism (instance.__dict__, then instance.__class__.__dict__) for
> some reason.
>
> >>> Foo.x = 1  # stored in class __dict__
> >>> f.x
> 1
> >>> f.__dict__['x'] = 2  # stored in instance __dict__
> >>> f.x
> 2
> >>> Foo.x
>
> 1
>
> But __class__ doesn't behave like this. Why?
>
> --
> Steven.

Perhaps because __class__ is a special descriptor:

>>> type(object.__dict__['__class__'])


George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bypassing __setattr__ for changing special attributes

[George Sakkis]

On Feb 20, 3:54 am, "[EMAIL PROTECTED]"
<[EMAIL PROTECTED]> wrote:
> On 20 fév, 05:39, "George Sakkis" <[EMAIL PROTECTED]> wrote:
>
> > I was kinda surprised that setting __class__ or __dict__ goes through
> > the __setattr__ mechanism, like a normal attribute:
>
> But __class__ and __dict___ *are* 'normal' attributes...

Well, let alone for the fact that two leading and trailing underscores
means 'special' in python, these attributes are not exactly what they
seem to be, i.e. a 'normal' type or dict:

>>> type(object.__dict__['__class__'])

>>> type(type.__dict__['__dict__'])



George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting directory contents

[Jussi Salmela]

Wolfgang Draxinger kirjoitti:
> H folks,
> 
> I got, hmm not really a problem, more a question of elegance:
> 
> In a current project I have to read in some files in a given
> directory in chronological order, so that I can concatenate the
> contents in those files into a new one (it's XML and I have to
> concatenate some subelements, about 4 levels below the root
> element). It all works, but somehow I got the feeling, that my
> solution is not as elegant as it could be:
> 
> src_file_paths = dict()
> for fname in os.listdir(sourcedir):
> fpath = sourcedir+os.sep+fname
> if not match_fname_pattern(fname): continue
> src_file_paths[os.stat(fpath).st_mtime] = fpath
> for ftime in src_file_paths.keys().sort():
> read_and_concatenate(src_file_paths[ftime])
> 
> of course listdir and sorting could be done in a separate
> function, but I wonder if there was a more elegant approach.
> 
> Wolfgang Draxinger

I'm not claiming the following to be more elegant, but I would do it 
like this (not tested!):

src_file_paths = dict()
prefix = sourcedir + os.sep
for fname in os.listdir(sourcedir):
 if match_fname_pattern(fname):
 fpath = prefix + fname
 src_file_paths[os.stat(fpath).st_mtime] = fpath
for ftime in src_file_paths.keys().sort():
 read_and_concatenate(src_file_paths[ftime])


Cheers,
Jussi
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting directory contents

[Wolfgang Draxinger]

Jussi Salmela wrote:

> I'm not claiming the following to be more elegant, but I would
> do it like this (not tested!):
> 
> src_file_paths = dict()
> prefix = sourcedir + os.sep
> for fname in os.listdir(sourcedir):
>  if match_fname_pattern(fname):
>  fpath = prefix + fname
>  src_file_paths[os.stat(fpath).st_mtime] = fpath
> for ftime in src_file_paths.keys().sort():
>  read_and_concatenate(src_file_paths[ftime])

Well, both versions, mine and yours won't work as it was written
down, as they neglegt the fact, that different files can have
the same st_mtime and that .sort() doesn't return a
sorted list.

However this code works (tested) and behaves just like listdir,
only that it sorts files chronologically, then alphabetically.

def listdir_chrono(dirpath):
import os
files_dict = dict()
for fname in os.listdir(dirpath):
mtime = os.stat(dirpath+os.sep+fname).st_mtime
if not mtime in files_dict:
files_dict[mtime] = list()
files_dict[mtime].append(fname)

mtimes = files_dict.keys()
mtimes.sort()
filenames = list()
for mtime in mtimes:
fnames = files_dict[mtime]
fnames.sort()
for fname in fnames:
filenames.append(fname)
return filenames

Wolfgang Draxinger
-- 
E-Mail address works, Jabber: [EMAIL PROTECTED], ICQ: 134682867

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting directory contents

[Larry Bates]

Wolfgang Draxinger wrote:
> Jussi Salmela wrote:
> 
>> I'm not claiming the following to be more elegant, but I would
>> do it like this (not tested!):
>>
>> src_file_paths = dict()
>> prefix = sourcedir + os.sep
>> for fname in os.listdir(sourcedir):
>>  if match_fname_pattern(fname):
>>  fpath = prefix + fname
>>  src_file_paths[os.stat(fpath).st_mtime] = fpath
>> for ftime in src_file_paths.keys().sort():
>>  read_and_concatenate(src_file_paths[ftime])
> 
> Well, both versions, mine and yours won't work as it was written
> down, as they neglegt the fact, that different files can have
> the same st_mtime and that .sort() doesn't return a
> sorted list.
> 
> However this code works (tested) and behaves just like listdir,
> only that it sorts files chronologically, then alphabetically.
> 
> def listdir_chrono(dirpath):
> import os
> files_dict = dict()
> for fname in os.listdir(dirpath):
> mtime = os.stat(dirpath+os.sep+fname).st_mtime
> if not mtime in files_dict:
> files_dict[mtime] = list()
> files_dict[mtime].append(fname)
> 
> mtimes = files_dict.keys()
> mtimes.sort()
> filenames = list()
> for mtime in mtimes:
> fnames = files_dict[mtime]
> fnames.sort()
> for fname in fnames:
> filenames.append(fname)
> return filenames
> 
> Wolfgang Draxinger

Four suggestions:

1) You might want to use os.path.join(dirpath, fname) instead of
   dirpath+os.sep+fname.

2) You may be able to use glob.glob() to filter the files
   more easily.

3) You didn't handle the possibility that there is s subdirectory
   in the current directory.  You need to check to make sure it is
   a file you are processing as os.listdir() returns files AND
   directories.

4) If you just put a tuple containing (mtime, filename) in a list
   each time through the loop you can just sort that list at the
   end it will be sorted by mtime and then alphabetically.

Example (not tested):

def listdir_chrono(dirpath):
import os
#
# Get a list of full pathnames for all the files in dirpath
# and exclude all the subdirectories.  Note: This might be
# able to be replaced by glob.glob() to simplify.  I would then
# add a second optional parameter: mask="" that would allow me
# to pass in a mask.
#
# List comprehensions are our friend when we are processing
# lists of things.
#
files=[os.path.join(dirpath, x) for x in os.listdir(dirpath)
   if not os.path.isdir(os.path.join(dirpath, x)]

#
# Get a list of tuples that contain (mtime, filename) that
# I can sort.
#
flist=[(os.stat(x).st_mtime, x) for x in files]

#
# Sort them.  Sort will sort on mtime, then on filename
#
flist.sort()
#
# Extract a list of the filenames only and return it
#
return [x[1] for x in flist]
#
# or if you only want the basenames of the files
#
#return [os.path.basename(x[1]) for x in flist]



-Larry Bates

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird result returned from adding floats depending on order I add them

[Grant Edwards]

On 2007-02-20, joanne matthews (RRes-Roth) <[EMAIL PROTECTED]> wrote:

> I'm getting different results when I add up a list of floats depending
> on the order that I list the floats.

That's how floats work.

> For example, the following returns
> False:
> def check():
>   totalProp=0
>   inputs=[0.2,0.2,0.2,0.1,0.2,0,0.1]
>   for each in inputs:
> 
>   totalProp+=each
>   print "totalProp=",totalProp
>   if totalProp != 1:

Floating point operations are not exact.  This test requires
them to be.

[...]

> Can anyone tell me whats going on

IEEE floating point can not exactly represent 0.2 nor 0.1, so
you get approximations.

> and how I can avoid the problem.

Don't use floating point if you expect exact results.

-- 
Grant Edwards   grante Yow!  I feel better about
  at   world problems now!
   visi.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: file io (lagged values) newbie question

[John Machin]

On Feb 21, 12:22 am, Steven D'Aprano
<[EMAIL PROTECTED]> wrote:
> On Mon, 19 Feb 2007 22:17:42 -0800, hiro wrote:
> > Hey there, I'm currently doing data preprocessing (generating lagged
> > values for a time series) and I'm having some difficulties trying to
> > write a file to disk.  A friend of mine, wrote this quick example for
> > me:
>
> If that's a quick example (well over 100 lines), I'd hate to see your
> idea of a long example.
>
> Can you cut out all the irrelevant cruft and just show:
>
> (1) the actual error you are getting
> (2) the SMALLEST amount of code that demonstrates the error
>
> Try to isolate if the problem is in *writing* the file or *generating*
> the time series.
>
> Hint: instead of one great big lump of code doing everything, write AT
> LEAST two functions: one to read values from a file and generate a time
> series, and a second to write it to a file.
>
> That exercise will probably help you discover what the problem is, and
> even if it doesn't, you'll have narrowed it down from one great big lump
> of code to a small function.
>
> To get you started, here's my idea for the second function:
> (warning: untested)
>
> def write_series(data, f):
> """Write a time series data to file f.
>
> data should be a list of integers.
> f should be an already opened file-like object.
> """
> # Convert data into a string for writing.
> s = str(data)
> s = s[1:-1]  # strip the leading and trailing [] delimiters
> s = s.replace(',', '') # delete the commas
> # Now write it to the file object
> f.write(s)
> f.write('\n')

And that's not cruft?

Try this: f.write(' '.join(str(x) for x in data) + '\n')

> And this is how you would use it:
>
> f = file('myfile.txt', 'w')
> # get some time series data somehow...
> data = [1, 2, 3, 4, 5]  # or something else
> write_series(data, f)
> # get some more data
> data = [2, 3, 4, 5, 6]
> write_series(data, f)
> # and now we're done
> f.close()
>

Or for a more general solution, use the csv module:

C:\junk>\python25\python
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import csv
>>> wtr = csv.writer(open('fubar.txt', 'wb'), delimiter=' ')
>>> data = [0, 1, 42, 666]
>>> wtr.writerow(data)
>>> wtr.writerow([9, 8, 7, 6])
>>> ^Z

C:\junk>type fubar.txt
0 1 42 666
9 8 7 6

HTH,
John

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting directory contents

[Paul Rubin]

Wolfgang Draxinger <[EMAIL PROTECTED]> writes:
> src_file_paths = dict()
> for fname in os.listdir(sourcedir):
> fpath = sourcedir+os.sep+fname
> if not match_fname_pattern(fname): continue
> src_file_paths[os.stat(fpath).st_mtime] = fpath
> for ftime in src_file_paths.keys().sort():
> read_and_concatenate(src_file_paths[ftime])

Note you have to used sorted() and not .sort() to get back a value
that you can iterate through.

Untested:

  from itertools import ifilter

  goodfiles = ifilter(match_fname_pattern,
  (sourcedir+os.sep+fname for \
 fname in os.listdir(sourcedir))

  for f,t in sorted((fname,os.stat(f).st_mtime) for fname in goodfiles,
key=lambda (fname,ftime): ftime):
 read_and_concatenate(f)

If you're a lambda-phobe you can use operator.itemgetter(1) instead of
the lambda.  Obviously you don't need the separate goodfiles variable
but things get a bit deeply nested without it.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird result returned from adding floats depending on order I add them

[John Machin]

On Feb 21, 2:05 am, Grant Edwards <[EMAIL PROTECTED]> wrote:
> On 2007-02-20, joanne matthews (RRes-Roth) <[EMAIL PROTECTED]> wrote:
>
> > I'm getting different results when I add up a list of floats depending
> > on the order that I list the floats.

>
> Don't use floating point if you expect exact results.

It's not the floating point that's the problem, it's the radix, in
this case 2, not being able to express n/10 exactly. As the tutorial
points out, radix-10 has problems representing n/3 (and n/7 and ...)
exactly.

Another take: Don't expect exact results. If the input is exact to 1
or two decimal places, don't expect the sum to be exact to 15 or more
decimal places.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting directory contents

[Jussi Salmela]

Wolfgang Draxinger kirjoitti:
> Jussi Salmela wrote:
> 
>> I'm not claiming the following to be more elegant, but I would
>> do it like this (not tested!):
>>
>> src_file_paths = dict()
>> prefix = sourcedir + os.sep
>> for fname in os.listdir(sourcedir):
>>  if match_fname_pattern(fname):
>>  fpath = prefix + fname
>>  src_file_paths[os.stat(fpath).st_mtime] = fpath
>> for ftime in src_file_paths.keys().sort():
>>  read_and_concatenate(src_file_paths[ftime])
> 
> Well, both versions, mine and yours won't work as it was written
> down, as they neglegt the fact, that different files can have
> the same st_mtime and that .sort() doesn't return a
> sorted list.
> 
> However this code works (tested) and behaves just like listdir,
> only that it sorts files chronologically, then alphabetically.
> 
> def listdir_chrono(dirpath):
> import os
> files_dict = dict()
> for fname in os.listdir(dirpath):
> mtime = os.stat(dirpath+os.sep+fname).st_mtime
> if not mtime in files_dict:
> files_dict[mtime] = list()
> files_dict[mtime].append(fname)
> 
> mtimes = files_dict.keys()
> mtimes.sort()
> filenames = list()
> for mtime in mtimes:
> fnames = files_dict[mtime]
> fnames.sort()
> for fname in fnames:
> filenames.append(fname)
> return filenames
> 
> Wolfgang Draxinger

More elegant or not ... I did it MY WA!!! (and tested this time 
really carefully;)):

#---
def listdir_chrono_2(dirpath):
 import os
 files_dict = {}
 prefix = dirpath + os.sep
 for fname in os.listdir(dirpath):
 mtime = os.stat(prefix + fname).st_mtime
 files_dict.setdefault(mtime, []).append(fname)

 mtimes = sorted(files_dict.keys())
 filenames = []
 for mtime in mtimes:
 filenames += sorted(files_dict[mtime])
 return filenames

firstLst = listdir_chrono('.')
secondLst = listdir_chrono_2('.')
if firstLst == secondLst: print 'OK'
else: print 'ERROR!!!'
#---

I keep taking the "dirpath + os.sep" part out of the loop because it is 
a loop invariant and doesn't have to be inside the loop.

Cheer
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxPython: non-GUI thread launching new frame? Delegates?

[Chris Mellon]

On 2/20/07, Diez B. Roggisch <[EMAIL PROTECTED]> wrote:
> cyberco wrote:
>
> > In my wxPython app a non-GUI thread (that reads info from the network)
> > tries to open a frame to show the new info. This results in my app
> > hanging (which is not too surprising). Coming from a C# environment I
> > wonder if there is some sort of delegate mechanism in wxPython to do
> > this sort of thing.
>
> Not sure how wx deals with this, but one thing you might explore is the
> possibility to add a timer in the GUI-thread, that polls a thread-filled
> queue.
>
> Other toolkits as Qt have means to insert an extra event in the event queue
> of the gui-thread in a thread-agnostic way, maybe wx has that too.
>
> Googling...
> ...
> ...
> ...
>
> ... finished
>
> http://mail.python.org/pipermail/python-list/2005-August/335467.html
>
> """
> You need another way to pass completion information between the downloader
> thread and the main thread; the simplest way is to define a custom wx
> Event, and wxPostEvent from the downloader thread when it completes (
> and when the gauge should be updated).  wxPostEvent is safe to call from
> non-eventloop threads.
> The main thread's wx event loop just spins, properly updating all other
> parts of the GUI, and receiving events from the downloader thread.
>
> ANother approach is to have a thread-safe Queue and have the main
> thread/event loop
> poll the queue with queue.get_nowait() periodically (typically 0.1-1 sec).
> The downloader thread shares the queue object and puts data structures
> (typically
> class instances, strings, or ints) that indicate status updates.
> """
>
> So - both options a viable. And read to the end, the twisted-approach
> certainly is the most clean one.
>

This is rather out of date. wxPython provides a wx.CallAfter function,
which will call the passed callable on the next spin through the event
loop. It's essentially a wrapper around the custom event mechanism
described above.

> Diez
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting directory contents

[Rob Wolfe]

Wolfgang Draxinger wrote:

> However this code works (tested) and behaves just like listdir,
> only that it sorts files chronologically, then alphabetically.
>
> def listdir_chrono(dirpath):
> import os
> files_dict = dict()
> for fname in os.listdir(dirpath):
> mtime = os.stat(dirpath+os.sep+fname).st_mtime
> if not mtime in files_dict:
> files_dict[mtime] = list()
> files_dict[mtime].append(fname)
>
> mtimes = files_dict.keys()
> mtimes.sort()
> filenames = list()
> for mtime in mtimes:
> fnames = files_dict[mtime]
> fnames.sort()
> for fname in fnames:
> filenames.append(fname)
> return filenames

Using the builtin functions `sorted`, `filter` and the `setdefault`
method of dictionary could a little shorten your code:

def listdir_chrono(dirpath):
import os
files_dict = {}
for fname in filter(os.path.isfile, os.listdir(dirpath)):
mtime = os.stat(os.path.join(dirpath, fname)).st_mtime
files_dict.setdefault(mtime, []).append(fname)

filenames = []
for mtime in sorted(files_dict):
for fname in sorted(files_dict[mtime]):
filenames.append(fname)
return filenames

--
HTH,
Rob

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python-list Digest, Vol 41, Issue 286

[Paul Boddie]

On 20 Feb, 11:59, KoDer <[EMAIL PROTECTED]> wrote:
> > Subject: Can I reverse eng a .pyc back to .py?

[...]

> https://svn.cs.pomona.edu/sys/src/konane/pyunparse.py
>
> Need add new AST nodes if You want to use it with python2.5.

The above code seems to need to parse a source file, since it uses the
compiler module's parse function. Something like decompyle would be
more appropriate for reverse engineering purposes:

http://packages.qa.debian.org/d/decompyle.html

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Found a product for running Python-based websites off CDROM -have anybody tried it?

[Don Taylor]

David Wishnie wrote:
> Hello,
> 
> Recently I've found a product that allows to create CDs or DVDs with 
> mod_python  -based websites
> (and CGI python of  course) so that apache-based webserver, python and 
> mod_python are run directly
> off CD on  Windows, MacOS X  and Linux at the same time (also it seems 
> to support perl, java,
> php and mysql + SQLite as databases).
> 
> http://www.stunnix.com/prod/aws/overview.shtml
> 
> Have anybody tried it? I'm considering to use it for several projects.
> 
> Thanks,
>   David
> 
That is an expensive product ($789) especially considering it mostly 
consists of FOSS pieces.

I found XAMPP, a FOSS, that is almost the same thing:

http://portableapps.com/apps/development/xampp

and this thread for getting mod_python running (scroll down a bit to the 
second post):

http://www.apachefriends.org/f/viewtopic.php?t=21169&highlight=python

I have not tried this yet.

Don.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: builtin set literal

[bearophileHUGS]

Steven Bethard:
> While Python 3.0 is not afraid to break backwards
> compatibility, it tries to do so only when there's a very substantial
> advantage.

I understand, but this means starting already to put (tiny)
inconsistencies into Python 3.0...

Unrelated: Ruby and Lisp use ? and ! at the end of the function/method
names to denote a predicate or a function that mutates in place (With
them the list.sort() may be called list.sort!() ). Using Python I
usually put an Q at the end of the name for this purpose. Can Py 3.0
support names ending with "?" too?

Bye,
bearophile

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: file io (lagged values) newbie question

[Steven D'Aprano]

On Tue, 20 Feb 2007 07:08:02 -0800, John Machin wrote:

>> def write_series(data, f):
>> """Write a time series data to file f.
>>
>> data should be a list of integers.
>> f should be an already opened file-like object.
>> """
>> # Convert data into a string for writing.
>> s = str(data)
>> s = s[1:-1]  # strip the leading and trailing [] delimiters
>> s = s.replace(',', '') # delete the commas
>> # Now write it to the file object
>> f.write(s)
>> f.write('\n')
> 
> And that's not cruft?

No. Why do you think it is crufty?

Would it be less crufty if I wrote it as a cryptic one liner without
comments?

f.write(str(data)[1:-1].replace(',', '') + '\n')

Okay, it depends on the string conversion of a list. But that's not going
to change any time soon.

> Try this: f.write(' '.join(str(x) for x in data) + '\n')

That will only work in Python 2.5 or better.


-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: builtin set literal

[Paul Rubin]

[EMAIL PROTECTED] writes:
> Unrelated: Ruby and Lisp use ? and ! at the end of the function/method
> names to denote a predicate or a function that mutates in place (With
> them the list.sort() may be called list.sort!() ). Using Python I
> usually put an Q at the end of the name for this purpose. Can Py 3.0
> support names ending with "?" too?

I don't think that's part of the plan.  However, Python seems to use
the -ed suffix for the non-mutating versions of these functions, e.g.
sorted(list) instead of the mutating list.sort().  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: file io (lagged values) newbie question

[Paul Rubin]

Steven D'Aprano <[EMAIL PROTECTED]> writes:
> Would it be less crufty if I wrote it as a cryptic one liner without
> comments?
> 
> f.write(str(data)[1:-1].replace(',', '') + '\n')

That doesn't look terribly cryptic to me, but maybe I'm used to it.

> > Try this: f.write(' '.join(str(x) for x in data) + '\n')
> That will only work in Python 2.5 or better.

It should work in 2.4, I think.  Though, I haven't tried 2.5 yet so I
can't say from experience whether 2.4 is better than 2.5.  Anyway,

  f.write(' '.join(map(str, data)) + '\n')

should work in the whole 2.x series, if I'm not mistaken.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird result returned from adding floats depending on order I add them

[Grant Edwards]

On 2007-02-20, John Machin <[EMAIL PROTECTED]> wrote:
> On Feb 21, 2:05 am, Grant Edwards <[EMAIL PROTECTED]> wrote:
>> On 2007-02-20, joanne matthews (RRes-Roth) <[EMAIL PROTECTED]> wrote:
>>
>> > I'm getting different results when I add up a list of floats depending
>> > on the order that I list the floats.
>
>>
>> Don't use floating point if you expect exact results.
>
> It's not the floating point that's the problem, it's the radix, in
> this case 2, not being able to express n/10 exactly. As the tutorial
> points out, radix-10 has problems representing n/3 (and n/7 and ...)
> exactly.

No matter what radix you choose, you're going to be able to
exactly represent 0% of the rational numbers within the range
of the representation.  Since you have no control over the FP
representation (and hence radix), and little control over input
values, the only sane thing to do is to write your code under
the assumption that FP can't represent any values exactly.

> Another take: Don't expect exact results.

Which is what I said.  :)

> If the input is exact to 1 or two decimal places, don't expect
> the sum to be exact to 15 or more decimal places.

In this case the input values have about 14 significant digits.
So does the output.  Unfortunately, the algorithm as written
requires an infinite number of significant digits.

-- 
Grant Edwards   grante Yow!  WHOA!! Ken and
  at   Barbie are having TOO
   visi.comMUCH FUN!! It must be the
   NEGATIVE IONS!!
-- 
http://mail.python.org/mailman/listinfo/python-list

Python 3.0 unfit for serious work?

[Jay Tee]

Yo,

On Feb 16, 6:07 am, Steven Bethard <[EMAIL PROTECTED]> wrote:
> Python 3.0 is determined not to be hampered by backwards incompatibility
> concerns. It's not even clear yet that your average 2.6 code will work

Then Python is pretty much determined to remove itself from
consideration
from various kinds of international projects like the one I work on.
We're already catching flack from people due to a few things that were
valid
in 2.2 that are not valid in 2.3 (I don't have the details but could
scare them
up).  The project we work on contains code from many different people
and has to
run on thousands of computers all over the world.  The installed base
at the
moment is a mix of RHEL 3, RHEL 4, and Debian, with a few other
machines thrown in.
The relevant Python versions at this moment IIRC are 2.2.3 and 2.3.4,
because these
are the native versions on those platforms.

We are estimating, due to the speed at which our applications follow
OS releases, that
we can drop RHEL 3 (and hence Python 2.2) support a year from now.  Go
figure when you
think we might be ready to require that all programs run on python
3.0.  If it's not
backwards compatible, meaning if 2.4 code doesn't run on 3.0, it's
rather likely that
strong pressure will be applied to port *away* from Python into
something less capricious.

Bottom line: practicality and beauty is always a tradeoff.  Oberon is
the most beautiful
language I ever saw, but there is almost nobody using it any more.
Too many beauty contests over who had the best proposal for a standard
library.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if one dict is subset of another?

[Jay Tee]

On Feb 20, 9:12 am, Paul Rubin  wrote:
 >   do_something()
>
> you'd just write:
>
> for j in (index['user']['jeff'] & index['state']['running']):
> do_something()

Hi,

it looks very cool, except that one of the constraints mentioned is
that the solution has to work properly on pythons 2.2 and 2.3.
production systems, certified OS releases, that sort of stuff ...
can't just upgrade a few thousand machines worldwide for my little
program ;-)

Thanks ... I took the previous poster's suggestion (had also been
suggested earlier) and built cached versions of the job lists, indexed
on common queries, while building the master job list.

JT

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if one dict is subset of another?

[Paul Rubin]

"Jay Tee" <[EMAIL PROTECTED]> writes:
> it looks very cool, except that one of the constraints mentioned is
> that the solution has to work properly on pythons 2.2 and 2.3.

That thing doesn't really deeply depend on defaultdict, it's just
convenient.  You can add a few more lines of code in the preprocessing
step to get around it.

> Thanks ... I took the previous poster's suggestion (had also been
> suggested earlier) and built cached versions of the job lists, indexed
> on common queries, while building the master job list.

You can also cache queries with the scheme I suggested, and it lets
you make the cached lists for new queries quickly.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if one dict is subset of another?

[Jay Tee]

Hi

your post had the following construct:

> for j in (index['user']['jeff'] & index['state']['running']):
> do_something()

but

Python 2.3.4 (#1, Oct 11 2006, 06:18:43)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> l1= [3, 4, 7, 2]
>>> l2 = [2, 3]
>>> l2 = [2, 3, 99]
>>> l1 & l2
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: unsupported operand type(s) for &: 'list' and 'list'

what am I missing?

Thanks

JT

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if one dict is subset of another?

[Paul Rubin]

"Jay Tee" <[EMAIL PROTECTED]> writes:
> >>> l1= [3, 4, 7, 2]
> >>> l2 = [2, 3]
> >>> l2 = [2, 3, 99]
> >>> l1 & l2
> Traceback (most recent call last):
>   File "", line 1, in ?
> TypeError: unsupported operand type(s) for &: 'list' and 'list'
> 
> what am I missing?

They are sets, not lists.

   from sets import Set as set  # use in 2.3 and earlier

   l1= set([3, 4, 7, 2])
   l2 = set([2, 3])
   l2 = set([2, 3, 99])
   print l1 & l2
-- 
http://mail.python.org/mailman/listinfo/python-list

Regd. converting seconds to hh:mm:ss format

[Vishal Bhargava]

Is there an inbuilt library in Python which you can use to convert time in
seconds to hh:mm:ss format?
Thanks,
Vishal

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help Required for Choosing Programming Language

[Jordan]

On Feb 18, 7:35 pm, Dave Cook <[EMAIL PROTECTED]> wrote:
> On 2007-02-16, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
> > i have read about Python, Ruby and Visual C++. but i want to go
> > through with GUI based programming language like VB.net
>
> You might take a look athttp://dabodev.com
>
> Dave Cook

Learn python, and then learn wxpython (python binding of wxWidgets).
wxWidgets is one of the most extensive and well built gui tools (in my
opinion) and is easier to implement in python than C++ (also my
opinion).  The only reason I can see for using C++ over Python for gui
building is if for some obscure reason Python wasn't fast enough.  On
the other hand, this brings up the argument of which is faster: Python
or C++  ;)   so let's not get into that.  Python + WxPython = Good
Language with Good GUI Toolkit.

Cheers,
Jordan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Regd. converting seconds to hh:mm:ss format

[Steve Holden]

Vishal Bhargava wrote:
> Is there an inbuilt library in Python which you can use to convert time in
> seconds to hh:mm:ss format?
> Thanks,
> Vishal
> 
Please don't ask a question by editing a reply to an existing thread: 
your question is now filed on many people's computers under "How to test 
if one dict us a subset of another".

Look at the strftime() function in the time module.

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd  http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Blog of Note:  http://holdenweb.blogspot.com
See you at PyCon? http://us.pycon.org/TX2007
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Regd. converting seconds to hh:mm:ss format

[Diez B. Roggisch]

Vishal Bhargava schrieb:
> Is there an inbuilt library in Python which you can use to convert time in
> seconds to hh:mm:ss format?


Module datetime. Did you even _bother_ to google or skim the docs?

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ocaml to python

[Laurent Pointal]

Gigs_ wrote:

> Is there any way to convert ocaml code to python? but not manually

For Python and ocaml, my bookmark contains this link:

http://www.programs4all.net/programs/Pycaml-Python-Embedding-API-for-Ocaml.htm

But no ocaml to Python compiler...

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Regd. converting seconds to hh:mm:ss format

[Sick Monkey]

Do you mean epoch time?  if so, the below example would work.


import time,datetime
t = datetime.datetime.now()
print t

2007-02-20 13:09:34.851000

print "Epoch Seconds:", time.mktime(t.timetuple())

Epoch Seconds: 1171994974.0

# go the other way

epochtime = 1171994974
now = datetime.datetime.fromtimestamp(epochtime)
print now

2007-02-20 13:09:34

On 2/20/07, Vishal Bhargava <[EMAIL PROTECTED]> wrote:



Is there an inbuilt library in Python which you can use to convert time in
seconds to hh:mm:ss format?
Thanks,
Vishal

--
http://mail.python.org/mailman/listinfo/python-list

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CherryPy/Turbogears on server not controlled by me

[Eddie Corns]

"Diez B. Roggisch" <[EMAIL PROTECTED]> writes:

>Brian Blais wrote:

>> Hello,
>> 
>> I was wondering if there is a way to run CherryPy/Turbogears on a server
>> that I don't
>> have root access to.  If I just choose a random port, I think the security
>> guys on
>> the server would get annoyed at me.  

>Why should they? Opening anything networking will open a port. And if you
>have shell access, you can point PYTHONPATH to a directory of your choice
>and let easy_install install the packages required there.

>I don't see a problem. And _if_ they get mad at you, well - there isn't
>anything you can do about that I presume apart from telling them to shut
>up - because without a port, there is no webapp...

And since your application is not running as root the worst that can happen if
you introduce a security hazard in your app is that only your files are
endangered.  I run several TG apps in user space and the only small problem
I've had is the logistics of telling people if you have to move it somewhere
else since a DNS alias doesn't help much with explicit port numbers in the URL.

Eddie
-- 
http://mail.python.org/mailman/listinfo/python-list

PLY for standard library

[Alan Isaac]

Is PLY destined for the standard library?
If not, what module providing substantially similar functionality is?

Thank you,
Alan Isaac


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: threading and multicores, pros and cons

[Nikita the Spider]

In article <[EMAIL PROTECTED]>,
 Paul Rubin  wrote:

> Nikita the Spider <[EMAIL PROTECTED]> writes:
> > note, there a (sort of) new module available that allows interprocess 
> > communication via shared memory and semaphores with Python. You can find 
> > it here: 
> > http://NikitaTheSpider.com/python/shm/
> 
> This is from the old shm module that was floating around several years
> ago?  Cool, I remember trying to find it recently and it seemed to
> have disappeared--the original url was dead and it wasn't mirrored
> anywhere. 

Yes, this is almost certainly the one which you remember. I had a hard 
time finding it myself, but it's still shipped with a few Linux distros 
that have their SVN repository online and indexed by Google. 

FYI, I fixed a few bugs in the original, added some small features and a 
wrapper module. If you're compiling for Linux you might need to remove 
the HAVE_UNION_SEMUN definition from setup.py. (Just learned this 
yesterday thanks to Eric J. and I haven't updated the documentation yet.)

> How about putting it in CheeseShop or some other such repository?  

Hmmm, I hadn't thought about that since I've never used the Cheese Shop 
myself. What benefits does Cheese 
Shop confer to someone looking for a package? I ask because from my 
perspective it just adds overhead to package maintenance. 

> Having it in the stdlib would be even better, of course.

That'd be fine with me!

-- 
Philip
http://NikitaTheSpider.com/
Whole-site HTML validation, link checking and more
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pep 3105: the end of print?

[Beliavsky]

On Feb 16, 10:17 am, Steven D'Aprano
<[EMAIL PROTECTED]> wrote:
> On Fri, 16 Feb 2007 09:49:03 -0500, Jean-Paul Calderone wrote:
> > On Sat, 17 Feb 2007 01:32:21 +1100, Steven D'Aprano
> >> [snip]
>
> >>I don't think that follows at all. print is only a problem if you expect
> >>your code to work under both Python 2.x and 3.x. I wouldn't imagine
> >>that many people are going to expect that: I know I don't.
>
> > I think some people are confused that the language "Python 3.x" has "Python"
> > in its name, since there is already a language with "Python" in its name,
> > with which it is not compatible.
>
> There is no language "Python 3" -- yet. We're still all guessing just
> how compatible it will be with Python 2.
>
> To be precise, there is an experimental Python 3, but the specifications
> of the language are not fixed yet.
>
> As for the name... big deal. C and C++ and Objective C are completely
> different languages.Fortran 77 and Fortran 90 aren't exactly the same;
> nobody expects the same code to run unmodified on Forth 77 and FigForth,
> or any of another three dozen varieties of Forth.

The attitudes towards backwards compatibility of the Fortran standards
committee and the Python BDFL is very different.

Fortran 90 was a superset of Fortran 77, so standard-conforming
Fortran 77 programs have the same meaning when compiled with an F90
compiler. The committee does not consider changing things like the
meaning of 1/2. Even though WRITE provides a superset of the
functionality of PRINT, the committee would not dream of breaking so
much code by removing PRINT. No feature is deleted without being
declared obsolescent in a previous standard, which makes it easier to
plan ahead. In practice Fortran 95 compilers accept deleted features,
but they are required to a have a switch that identifies non-standard
code.

I think the C and C++ committees also take backwards compatibility
seriously, in part because they know
that working programmers will ignore them if they break too much old
code.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: builtin set literal

[Alan Isaac]

Paul Rubin wrote:
> There's even a sentiment in some pythonistas to get rid of the [] and {}
> notations for lists and dicts, using list((1,2,3)) and dict((1,2),(3,4))
> for [1,2,3] and {1:2, 3:4} respectively.

Well then for consistency they must want tuple((1,2,3)) for (1,2,3).
Oh oh, that must be tuple(tuple((1,2,3))), no wait ...

Alan Isaac


-- 
http://mail.python.org/mailman/listinfo/python-list

question with inspect module

[Tool69]

Hi,

I would like to retrieve all the classes, methods and functions of a
module.
I've used the inspect module for this, but inside a given class
(subclass of some other one), I wanted to retrieve only the methods
I've written, not the inherited one. How can I do ?

Thanks.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PLY for standard library

[Fuzzyman]

On Feb 20, 6:29 pm, "Alan Isaac" <[EMAIL PROTECTED]> wrote:
> Is PLY destined for the standard library?

I've never heard it suggested... but +1

Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml

> If not, what module providing substantially similar functionality is?
>
> Thank you,
> Alan Isaac


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting directory contents

[Wolfgang Draxinger]

Larry Bates wrote:

> 3) You didn't handle the possibility that there is s
> subdirectory
>in the current directory.  You need to check to make sure it
>is a file you are processing as os.listdir() returns files
>AND directories.

Well, the directory the files are in is not supposed to have any
subdirectories.

> 4) If you just put a tuple containing (mtime, filename) in a
> list
>each time through the loop you can just sort that list at
>the end it will be sorted by mtime and then alphabetically.

Ah, of course. Hmm, seems I was short of caffeine when I hacked
my code :-P

> def listdir_chrono(dirpath):
> import os
> #
> # Get a list of full pathnames for all the files in dirpath
> # and exclude all the subdirectories.  Note: This might be
> # able to be replaced by glob.glob() to simplify.  I would
> # then add a second optional parameter: mask="" that would
> # allow me to pass in a mask.
> #
> # List comprehensions are our friend when we are processing
> # lists of things.
> #
> files=[os.path.join(dirpath, x) for x in
> os.listdir(dirpath)
>if not os.path.isdir(os.path.join(dirpath, x)]
> 
> #
> # Get a list of tuples that contain (mtime, filename) that
> # I can sort.
> #
> flist=[(os.stat(x).st_mtime, x) for x in files]
> 
> #
> # Sort them.  Sort will sort on mtime, then on filename
> #
> flist.sort()
> #
> # Extract a list of the filenames only and return it
> #
> return [x[1] for x in flist]
> #
> # or if you only want the basenames of the files
> #
> #return [os.path.basename(x[1]) for x in flist]

Now, THAT is elegant.

Wolfgang Draxinger
-- 
E-Mail address works, Jabber: [EMAIL PROTECTED], ICQ: 134682867

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if one dict is subset of another?

[Jay Tee]

On Feb 20, 6:44 pm, Paul Rubin  wrote:
> They are sets, not lists.
>
>from sets import Set as set  # use in 2.3 and earlier
>
>l1= set([3, 4, 7, 2])
>l2 = set([2, 3])
>l2 = set([2, 3, 99])
>print l1 & l2

Thanks Paul, but:

bosui:~> python
Python 2.2.3 (#1, Oct 26 2003, 11:49:53)
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-20)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from sets import Set as set
Traceback (most recent call last):
  File "", line 1, in ?
ImportError: No module named sets

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: builtin set literal

[Steven Bethard]

Steven Bethard:
 > While Python 3.0 is not afraid to break backwards
 > compatibility, it tries to do so only when there's a very substantial
 > advantage.

[EMAIL PROTECTED] wrote:
> I understand, but this means starting already to put (tiny)
> inconsistencies into Python 3.0...

Well, there's going to be an inconsistency one way or another:

Lists:
   [1, 2]
   [1]
   []

Dicts:
   {1:2, 2:1}
   {1:2}
   {}

Sets:
   {1, 2}
   {1}
   set()

Note that if we used {} for the empty set and {:} for the empty dict, 
then sets would be consistent, but dicts would be inconsistent. And if 
you're really worried about consistencies, take a look at the current 
state of tuples:

   1, 2
   1,
   ()

There's just not an obvious *right* answer here, so it's better to stick 
with the backwards compatible version.

STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: threading and multicores, pros and cons

[Paul Boddie]

Nikita the Spider wrote:
>
> Hmmm, I hadn't thought about that since I've never used the Cheese Shop
> myself. What benefits does Cheese
> Shop confer to someone looking for a package? I ask because from my
> perspective it just adds overhead to package maintenance.

The Python Package Index, as I prefer to call it (but we're talking
about the same thing), doesn't really make any special demands on
distribution or maintenance: you just need to register yourself and
add an entry for the package, filling in a few fields such as the
homepage and perhaps the download link; you can also upload archives
if you'd prefer. If you have a PKG-INFO file, you can either upload
that in order to get fields filled out more conveniently (as long as
the Package Index likes the file), and if you have a setup.py script
you might be able to use the upload feature with that (and the PKG-
INFO file, I suppose).

Don't be confused by all the setuptools extras and any insistence that
the Package Index works best with things that are packaged as Python
Eggs: whilst that might confer certain benefits, mostly to users who
rely on Egg dependencies, it's peripheral to the purpose of the
Package Index itself.

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if one dict is subset of another?

[Paul Rubin]

"Jay Tee" <[EMAIL PROTECTED]> writes:
> Python 2.2.3 (#1, Oct 26 2003, 11:49:53)
> ImportError: No module named sets

Hmm, well I think the sets module in 2.3 is written in Python, so you
could drop it into your application for use in 2.2.  Better would be
to use the C version from 2.4 if you can.  Or you could fake it with
dicts.  Sets are really just dicts under the skin.  Instead of
set([1,2,3]) you'd use {1:True, 2:True, 3:True}.  To intersect
two of them (untested):

def intersection(d1,d2):
   if len(d2) < len(d1):
 # swap them so d1 is the smaller one
 d1,d2 = d2,d1
   r = {}
   for k in d1.iterkeys():
 if k in d2:
   r[k] = True
   return r

The d1,d2 swap is just an optimization, since access is constant-time
you want to scan the smaller dictionary to minimize the number of
lookups.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[Steven Bethard]

Jay Tee wrote:
> Yo,
> 
> On Feb 16, 6:07 am, Steven Bethard <[EMAIL PROTECTED]> wrote:
>> Python 3.0 is determined not to be hampered by backwards incompatibility
>> concerns. It's not even clear yet that your average 2.6 code will work
> 
> Then Python is pretty much determined to remove itself from
> consideration
> from various kinds of international projects like the one I work on.

You snipped the rest of that comment:

"It's not even clear yet that your average 2.6 code will work on 3.0 -- 
though there's a pretty large contingent trying to make this true."

If you want to make sure average 2.6 code works on 3.0, please help 
contribute to Python. You can find out where best to focus your efforts 
by asking on python-dev:

 http://mail.python.org/mailman/listinfo/python-dev

> If it's not backwards compatible, meaning if 2.4 code doesn't run on
> 3.0, it's rather likely that strong pressure will be applied to port
> *away* from Python into something less capricious.

Well, Python 2.4 code will work on Python 2.6 and 2.7 so just because 
your code isn't yet compatible with Python 3.0 doesn't mean you should 
give up on Python.

Python 2.2 was released in early 2003 and you said you'd be dropping 
support for 2.2 in early 2008, so I conclude that since Python 2.5 was 
released in late 2006, you'll be ready to drop Python 2.5 support (and 
have 2.6/3.0 compatible code) by late 2011. Sure, it's a long way off, 
but you're writing 2.2 compatible code *now*. Is it really that bad to 
wait four years for Python 3.0?

STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: PyDSTool now compatible with numpy 1.0.1, scipy 0.5.2 and 64-bit CPUs.

[Rob Clewley]

We are pleased to announce version 0.84 of PyDSTool, an open-source
dynamical systems simulation, modeling, and analysis package.

This long-overdue release is primarily intended to bring existing
PyDSTool functionality up to date with the latest numpy and scipy releases
(previous versions required scipy 0.3.2, numarray, numeric, etc).
Also, PyDSTool is now compatible with 64-bit CPUs.

While we have added a few new features and made several fixes, major
improvements to functionality are in the pipeline for version 0.90.

Please see http://pydstool.sourceforge.net for release notes and documentation,
and http://sourceforge.net/projects/pydstool for downloading. As ever, please
send us feedback if you have any problems with this new release or ideas and
code contributions for future releases.

Regards,

  Rob, Erik, and Drew.
  Center for Applied Mathematics,
  Cornell University.

 **

PyDSTool is an integrated simulation, modeling and analysis package
for dynamical systems, written in Python (and partly in C). It is
being developed at Cornell University, and the source code is
available under the terms of the BSD license. PyDSTool runs on Linux,
Windows, and Macs, and aims to have a minimal number of package
dependencies.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxPython: non-GUI thread launching new frame? Delegates?

[cyberco]

Ah! Great tip, thanks!
Now instead of calling:

parent.onRequest(param)

I call:

wx.CallAfter(lambda x: parent.onRequest(x), param)

Way cool.
2B



> This is rather out of date. wxPython provides a wx.CallAfter function,
> which will call the passed callable on the next spin through the event
> loop. It's essentially a wrapper around the custom event mechanism
> described above.
>
> > Diez

-- 
http://mail.python.org/mailman/listinfo/python-list

How to do a Decorator Here?

[Gregory Piñero]

Need some decorator help.

I have a class.  And I want to add behavior to one of this class's
methods to be run before the class runs the actual method.  Is this
what decorators are for?

So the class I want to work with is string.Template

Let's say I have this:
from string import Template
a=Template("$var1 is a test")

def preprocess(var1):
   #Real code here will be more complicated, just an example
var1=var1.upper()

a.substitute(var1="greg")

So how can I have preprocess run before substitute is run?  I want the
user to be able to call a.substitute and have preprocess run
automatically.

Or is this not what decorators do?  I'm trying to avoid subclassing if I can.

Thanks in advance for the help.

-Greg
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxPython: non-GUI thread launching new frame? Delegates?

[Chris Mellon]

On 20 Feb 2007 12:01:02 -0800, cyberco <[EMAIL PROTECTED]> wrote:
> Ah! Great tip, thanks!
> Now instead of calling:
>
> parent.onRequest(param)
>
> I call:
>
> wx.CallAfter(lambda x: parent.onRequest(x), param)
>

You don't need the lambda - you can use:

wx.CallAfter(parent.OnRequest, param)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxPython: non-GUI thread launching new frame? Delegates?

[Grant Edwards]

On 2007-02-20, cyberco <[EMAIL PROTECTED]> wrote:
> Ah! Great tip, thanks!
> Now instead of calling:
>
> parent.onRequest(param)
>
> I call:
>
> wx.CallAfter(lambda x: parent.onRequest(x), param)

How does that differ from this?

  wx.CallAfter(parent.onRequest, param)

-- 
Grant Edwards   grante Yow!  .. Everything
  at   isFLIPPING AROUND!!
   visi.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sorting directory contents

[Tim Williams]

On 20/02/07, Wolfgang Draxinger <[EMAIL PROTECTED]> wrote:
> H folks,
>
> I got, hmm not really a problem, more a question of elegance:
>
> In a current project I have to read in some files in a given
> directory in chronological order, so that I can concatenate the
> contents in those files into a new one (it's XML and I have to
> concatenate some subelements, about 4 levels below the root
> element). It all works, but somehow I got the feeling, that my
> solution is not as elegant as it could be:
>
> src_file_paths = dict()
> for fname in os.listdir(sourcedir):
> fpath = sourcedir+os.sep+fname
> if not match_fname_pattern(fname): continue
> src_file_paths[os.stat(fpath).st_mtime] = fpath
> for ftime in src_file_paths.keys().sort():
> read_and_concatenate(src_file_paths[ftime])
>
> of course listdir and sorting could be done in a separate
> function, but I wonder if there was a more elegant approach.
>
> Wolfgang Draxinger
> --
> E-Mail address works, Jabber: [EMAIL PROTECTED], ICQ: 134682867

Are you running on windows?

>>> i,o,e = os.popen3("dir c:\windows /OD /A-D /B")
>>> [x.strip() for x in o.readlines() if 'a' in x ]

Gives a list of filenames (but not directories) in date order, that
match a pattern ('a' in x)

>>> i,o,e = os.popen3("dir c:\windows /O-D /A-D /B")
>>> [x.strip() for x in o.readlines() if 'a' in x]

Gives the same list but in reverse chronological order.

Something containing an approximation of my poor-writing style below
would do the job.
>>> i,o,e = os.popen3("dir c:\windows /OD /A-D /B")
>>> [os.path.join(a_path, x.strip()) for x in o.readlines() if
match_fname_pattern(x)]
['c:/windows/NeroDigital.ini', 'c:/windows/WindowsUpdate.log',
'c:/windows/setupapi.log', 'c:/windows/wiadebug.log',
'c:/windows/wiaservc.log' 

c:\> Dir /? will give you more help

HTH :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: file io (lagged values) newbie question

[Bruno Desthuilliers]

Steven D'Aprano a écrit :
> On Tue, 20 Feb 2007 07:08:02 -0800, John Machin wrote:
> 
> 
>>>def write_series(data, f):
>>>"""Write a time series data to file f.
>>>
>>>data should be a list of integers.
>>>f should be an already opened file-like object.
>>>"""
>>># Convert data into a string for writing.
>>>s = str(data)
>>>s = s[1:-1]  # strip the leading and trailing [] delimiters
>>>s = s.replace(',', '') # delete the commas
>>># Now write it to the file object
>>>f.write(s)
>>>f.write('\n')
>>
>>And that's not cruft?
> 
> 
> No. Why do you think it is crufty?

Because it is ?
 >
> Would it be less crufty if I wrote it as a cryptic one liner without
> comments?
> 
> f.write(str(data)[1:-1].replace(',', '') + '\n')

Nope. It's still a WTF.

> Okay, it depends on the string conversion of a list.

Nope. It depends on the *representation* of a list.

> But that's not going
> to change any time soon.

> 
>>Try this: f.write(' '.join(str(x) for x in data) + '\n')
> 
> 
> That will only work in Python 2.5 or better.

Really ?

Python 2.4.1 (#1, Jul 23 2005, 00:37:37)
[GCC 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2, pie-8.7.6)] on 
linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> " ".join(str(x) for x in range(10))
'0 1 2 3 4 5 6 7 8 9'
 >>>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: question with inspect module

[Miki]

Hello,

> I would like to retrieve all the classes, methods and functions of a
> module.
> I've used the inspect module for this, but inside a given class
> (subclass of some other one), I wanted to retrieve only the methods
> I've written, not the inherited one. How can I do ?

class A:
def a_method(self):
pass

def common_method(self):
pass

class B(A):
def common_method(self):
pass

def b_method(self):
pass

import inspect
from os.path import abspath

lines, start_line = inspect.getsourcelines(B)
end_line = start_line + len(lines)
filename = abspath(inspect.getsourcefile(B))

for name in dir(B):
method = getattr(B, name)
if not callable(method):
continue

lnum = method.func_code.co_firstlineno
fname = abspath(method.func_code.co_filename)

if (lnum >= start_line) and (lnum <= end_line) and (fname ==
filename):
print "%s is from B" % name

HTH,
--
Miki Tebeka <[EMAIL PROTECTED]>
http://pythonwise.blogspot.com

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do a Decorator Here?

[Goldfish]

On Feb 20, 8:20 pm, "Gregory Piñero" <[EMAIL PROTECTED]> wrote:
> Need some decorator help.
>
> I have a class.  And I want to add behavior to one of this class's
> methods to be run before the class runs the actual method.  Is this
> what decorators are for?
>
> So the class I want to work with is string.Template
>
> Let's say I have this:
> from string import Template
> a=Template("$var1 is a test")
>
> def preprocess(var1):
>#Real code here will be more complicated, just an example
> var1=var1.upper()
>
> a.substitute(var1="greg")
>
> So how can I have preprocess run before substitute is run?  I want the
> user to be able to call a.substitute and have preprocess run
> automatically.
>
> Or is this not what decorators do?  I'm trying to avoid subclassing if I can.
>
> Thanks in advance for the help.
>
> -Greg

That sounds like aspect oriented programming. Spring Python (http://
springpython.python-hosting.com) offers a way to wrap objects with
method interceptors. Method interceptors give you full control before
and after method calls.

Sometimes decorators can be used to do that as well, but they have
constraints in where they can be used. I found them too inflexible for
my needs, so I built an AOP solution.

Greg

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[BJörn Lindqvist]

What a load of bull crap. Python is one of the simplest packages to
have multiple version of installed. When Python 3.0 is released, all
Linux distros will acquire a symlink at /usr/bin/python2 pointing to
the latest Python 2.x version installed. Or something equivalent.
Rest assured that Linux distributors will not drop Python 2.x support
in the nearest decade. They are not stupid.

On 20 Feb 2007 08:49:10 -0800, Jay Tee <[EMAIL PROTECTED]> wrote:
> Yo,
>
> On Feb 16, 6:07 am, Steven Bethard <[EMAIL PROTECTED]> wrote:
> > Python 3.0 is determined not to be hampered by backwards incompatibility
> > concerns. It's not even clear yet that your average 2.6 code will work
>
> Then Python is pretty much determined to remove itself from
> consideration
> from various kinds of international projects like the one I work on.
> We're already catching flack from people due to a few things that were
> valid
> in 2.2 that are not valid in 2.3 (I don't have the details but could
> scare them
> up).  The project we work on contains code from many different people
> and has to
> run on thousands of computers all over the world.  The installed base
> at the
> moment is a mix of RHEL 3, RHEL 4, and Debian, with a few other
> machines thrown in.
> The relevant Python versions at this moment IIRC are 2.2.3 and 2.3.4,
> because these
> are the native versions on those platforms.
>
> We are estimating, due to the speed at which our applications follow
> OS releases, that
> we can drop RHEL 3 (and hence Python 2.2) support a year from now.  Go
> figure when you
> think we might be ready to require that all programs run on python
> 3.0.  If it's not
> backwards compatible, meaning if 2.4 code doesn't run on 3.0, it's
> rather likely that
> strong pressure will be applied to port *away* from Python into
> something less capricious.
>
> Bottom line: practicality and beauty is always a tradeoff.  Oberon is
> the most beautiful
> language I ever saw, but there is almost nobody using it any more.
> Too many beauty contests over who had the best proposal for a standard
> library.
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>


-- 
mvh Björn
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[Jay Tee]

Hi,

On Feb 20, 8:59 pm, Steven Bethard <[EMAIL PROTECTED]> wrote:
> You snipped the rest of that comment:
>
> "It's not even clear yet that your average 2.6 code will work on 3.0 --
> though there's a pretty large contingent trying to make this true."

Thanks for pointing this out.  I voted for the comp.lang.python
newsgroup back in the early 90's, my active days of python
'development' are over, it's all i can do to hang on to the code i've
been posting about.

> have 2.6/3.0 compatible code) by late 2011. Sure, it's a long way off,
> but you're writing 2.2 compatible code *now*. Is it really that bad to
> wait four years for Python 3.0?

As long as when python 3.0 shows up, i don't have to do a massive
rewrite.  I think I've really only had to change two or three things
over the years .. one was that I used to use words like "dict" and
"list" in my code, which broke in subtle ways when d = dict() became
legal.

I just dug out some code laying around on disk from 1994, and ran it,
unchanged, under python 2.3.5. If I can achieve this (running 2007
code under python3.0 in 2011 with no modifications), that'd be OK.

JT

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: question with inspect module

[Tool69]

Thanks Miki, that's exactly what I need :)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[Paul Boddie]

Steven Bethard wrote:
>
> Well, Python 2.4 code will work on Python 2.6 and 2.7 so just because
> your code isn't yet compatible with Python 3.0 doesn't mean you should
> give up on Python.

Perhaps the most important concern in the context of Python 3.0 is
what the term "Python" will come to mean to the different communities
using the language. Some people will be maintaining software that
needs to be compatible with older releases; these people aren't
technologically backwards: they just find that such older releases
provide sufficient capabilities for the implementation of their
solutions. For such people, "Python" will be the language they've
always used, with a gradual accumulation of features, some known
quirks, and some relatively minor incompatibility issues over the
years.

Meanwhile, the risk is that the core developers will only consider
Python 3.0 as "Python" and that people doing anything with older
releases will be on their own. If the gap is too wide between 2.x and
3.x, any lack of maintenance in the 2.x series will be perceived as an
abandonment of "Python" for certain kinds of user, and the result will
probably be a loss of confidence in both variants of the language.
Although I believe that some of the recent attempts to lower the
disruptiveness of Python 3.0 may have helped to maintain "Python" as a
single narrow continuum, I think some people overlook how fortunate
the relationship between Python's evolution and momentum has been, and
how easily such a relationship can be broken.

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[Jeff Templon]

yo,

Bjorn, I am not sure I see why my post is bull crap.  I think all you
are doing is agreeing with me.  My post was entitled "Python 3.0 unfit
for serious work", you just indicated that the Linux distros will
agree with me, in order to be taken seriously, the distros will have
to include 2.x python for a very long time.  If 3.0 and 2.x have any
serious degree of incompatibility, python will be a favorite subject
for religious rants and heated arguments for many people.  And if we
don't manage to restrain our developers from using features that force
us prematurely to move to 3.0 ... and don't underestimate what this
means, because this means other things will have to move as well,
which may have dependencies on yet other things like C++ library
versions ... then I would have to, for reasons of maintainability,
argue against continuing to allow python code development in the
project.  I love python, but not enough to make 20+ people's lives
difficult.

There are already people making this sort of argument in our project.

   JT

On 2/20/07, BJörn Lindqvist <[EMAIL PROTECTED]> wrote:
> What a load of bull crap. Python is one of the simplest packages to
> have multiple version of installed. When Python 3.0 is released, all
> Linux distros will acquire a symlink at /usr/bin/python2 pointing to
> the latest Python 2.x version installed. Or something equivalent.
> Rest assured that Linux distributors will not drop Python 2.x support
> in the nearest decade. They are not stupid.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do a Decorator Here?

[Gregory Piñero]

On 2/20/07, Tim Mitchell <[EMAIL PROTECTED]> wrote:
> Hi Greg,
>
> Decorators would work fine if the the class you were working with was
> _yours_ (ie. you wrote it), the problem here is that string.Template is
> someone else's class that you're trying to modify.
>
> Here's how a decorator would work (unfortunately it involves subclassing)
>
> class myTemplate(string.Template):
> def decorator(func):
> def preprocess(*args, **kwargs):
> # do preprocessing here
> kwargs['var1'] = "greg"
> # call actual method
> return func(*args, **kwargs)
> return preprocess
>
> @decorator
> def substitute(self, *args, **kwargs):
> return string.Template.substitute(self, *args, **kwargs)
>
> alternatively, if you don't mind changing the string.Template class
> itself (and are sure that it's not used anywhere else that doesn't
> require preprocessing) try:
>
> _substitute = string.Template.substitute
> def subs(self, *args, **kwargs):
> # do preprocessing here
> kwargs['var1'] = "greg"
> return _substitute(self, *args, **kwargs)
> string.Template.substitute = subs
>
> Cheers
> Tim
>

Thanks Tim.  that makes sense now.  Turns out my approach was too
complicated anyway.  I think I'm just going to use Cheetah :-)

But at least I know decorators a little better.

-Greg
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if one dict is subset of another?

[Jay Tee]

Hi,

thanks!  the code lift from 2.3 to 2.2 worked (thank Guido et al for
BACKWARDS COMPATIBILITY ;-)) ... unfortunately I was in a hurry to get
the release out since a colleague's cluster was croaking under the
load of the old, non-indexed version.  Your solution is nicer looking
than mine, and leads to a less complex implementation.  Rolling it
into the next release is going to have to wait a few weeks, I hope I
can remember it that long.  Thanks!!

JT

On Feb 20, 8:54 pm, Paul Rubin  wrote:
> "Jay Tee" <[EMAIL PROTECTED]> writes:
> > Python 2.2.3 (#1, Oct 26 2003, 11:49:53)
> > ImportError: No module named sets
>
> Hmm, well I think the sets module in 2.3 is written in Python, so you
> could drop it into your application for use in 2.2.  Better would be
> to use the C version from 2.4 if you can.  Or you could fake it with
> dicts.  Sets are really just dicts under the skin.  Instead of
> set([1,2,3]) you'd use {1:True, 2:True, 3:True}.  To intersect
> two of them (untested):
>
> def intersection(d1,d2):
>if len(d2) < len(d1):
>  # swap them so d1 is the smaller one
>  d1,d2 = d2,d1
>r = {}
>for k in d1.iterkeys():
>  if k in d2:
>r[k] = True
>return r
>
> The d1,d2 swap is just an optimization, since access is constant-time
> you want to scan the smaller dictionary to minimize the number of
> lookups.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird result returned from adding floats depending on order I add them

[John Machin]

On Feb 21, 3:44 am, Grant Edwards <[EMAIL PROTECTED]> wrote:
> On 2007-02-20, John Machin <[EMAIL PROTECTED]> wrote:
>
> > On Feb 21, 2:05 am, Grant Edwards <[EMAIL PROTECTED]> wrote:
> >> On 2007-02-20, joanne matthews (RRes-Roth) <[EMAIL PROTECTED]> wrote:
>
> >> Don't use floating point if you expect exact results.
>
>
> > Another take: Don't expect exact results.
>
> Which is what I said.  :)

It may well be what you said. I didn't hear that. What you wrote was
"Don't use floating point if you expect exact results." That is *not*
the same as "Don't expect exact results".



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: PyDSTool now compatible with numpy 1.0.1, scipy 0.5.2 and 64-bit CPUs.

[Stef Mientki]

Sounds GREAT !
thank you !
I just took a quick look,
the comparison to SimuLink looks good,
now if someone could make a comparison with Modelica ;-)

cheers,
Stef Mientki

Rob Clewley wrote:
> We are pleased to announce version 0.84 of PyDSTool, an open-source
> dynamical systems simulation, modeling, and analysis package.
> 
> This long-overdue release is primarily intended to bring existing
> PyDSTool functionality up to date with the latest numpy and scipy releases
> (previous versions required scipy 0.3.2, numarray, numeric, etc).
> Also, PyDSTool is now compatible with 64-bit CPUs.
> 
> While we have added a few new features and made several fixes, major
> improvements to functionality are in the pipeline for version 0.90.
> 
> Please see http://pydstool.sourceforge.net for release notes and 
> documentation,
> and http://sourceforge.net/projects/pydstool for downloading. As ever, 
> please
> send us feedback if you have any problems with this new release or ideas 
> and
> code contributions for future releases.
> 
> Regards,
> 
>  Rob, Erik, and Drew.
>  Center for Applied Mathematics,
>  Cornell University.
> 
> **
> 
> PyDSTool is an integrated simulation, modeling and analysis package
> for dynamical systems, written in Python (and partly in C). It is
> being developed at Cornell University, and the source code is
> available under the terms of the BSD license. PyDSTool runs on Linux,
> Windows, and Macs, and aims to have a minimal number of package
> dependencies.
-- 
http://mail.python.org/mailman/listinfo/python-list

Regex Speed

[garrickp]

While creating a log parser for fairly large logs, we have run into an
issue where the time to process was relatively unacceptable (upwards
of 5 minutes for 1-2 million lines of logs). In contrast, using the
Linux tool grep would complete the same search in a matter of seconds.

The search we used was a regex of 6 elements "or"ed together, with an
exclusionary set of ~3 elements. Due to the size of the files, we
decided to run these line by line, and due to the need of regex
expressions, we could not use more traditional string find methods.

We did pre-compile the regular expressions, and attempted tricks such
as map to remove as much overhead as possible.

With the known limitations of not being able to slurp the entire log
file into memory, and the need to use regular expressions, do you have
an ideas on how we might speed this up without resorting to system
calls (our current "solution")?

-- 
http://mail.python.org/mailman/listinfo/python-list

pass variables via url with python?

[Katrin Shafiepour]

Hi

I'm quite new to python and I'm trying to pass variables via url! was
wondering if you can help me?

ThanQ
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[Jay Tee]

Hi,

Paul, thanks for this, I didn't realize the scope of the situation.  I
agree with your assessment to the extent that I understand what the
whole python 3.0 thing is about.

Let's see if I can scare up something I wrote about ten years ago on a
now-dead language that I really wanted to use (wound up sticking with
python instead because "it was supported" ;-)

===
to figure out how to work things.  The fact that there are three (or
four depending if you count Linz V4) different Oberon System
implementations, and several different compilers, and even four or
five separate dialects of Oberon with none of them appearing to be
really "official", gives the impression of a fragmented, directionless
development effort, and a probability bordering on 1. that
whatever you try to do will be incompatible with all but a small
subset of what's available (unless you stick to writing small programs
like in the books.)  It does not matter if you tell people that this
is not so; something has to clearly stand out as being THE STANDARD
STUFF and all the other stuff as INTERESTING BUT NONTHREATENING SIDE
PROJECTS.  The STANDARD STUFF must include a sufficient number of
=

Oberon is really really cool, seriously, ... but nobody is using it.
People working on python development are of course free to do what
they want, but so are the users ...

   J "actie-reactie is what my ex-brother-in-law would say" T

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[Steven Bethard]

Jay Tee wrote:
> Let's see if I can scare up something I wrote about ten years ago on a
> now-dead language that I really wanted to use (wound up sticking with
> python instead because "it was supported" ;-)
> 
> ===
> to figure out how to work things.  The fact that there are three (or
> four depending if you count Linz V4) different Oberon System
> implementations, and several different compilers, and even four or
> five separate dialects of Oberon with none of them appearing to be
> really "official", gives the impression of a fragmented, directionless
> development effort, and a probability bordering on 1. that
> whatever you try to do will be incompatible with all but a small
> subset of what's available (unless you stick to writing small programs
> like in the books.)  It does not matter if you tell people that this
> is not so; something has to clearly stand out as being THE STANDARD
> STUFF and all the other stuff as INTERESTING BUT NONTHREATENING SIDE
> PROJECTS.  The STANDARD STUFF must include a sufficient number of
> =

Well, the problem of multiple standards shouldn't really apply in the 
same way to Python. Right now, Python 2.X is the standard. Python 2.6, 
2.7 and any later 2.X versions are intended to be transitional versions 
while the standard is migrating to Python 3.X. At some point, the 2.X 
line will almost certainly be discontinued.

So as a Python programmer, the path is clear. As soon as possible, you 
should make your code compatible with Python 3.0. That will likely mean 
taking advantage of some new features in Python 2.6, so "as soon as 
possible" may still mean many years for projects that need to support 
older versions of Python. Still, once Python 2.6 is installed everywhere 
by default, it shouldn't be difficult to start making code compatible 
with the new standard, Python 3.0.

STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[Paul Boddie]

Jay Tee wrote:
>
> Paul, thanks for this, I didn't realize the scope of the situation.  I
> agree with your assessment to the extent that I understand what the
> whole python 3.0 thing is about.

I don't know if I've delimited the scope of any situation, really.
However...

[...]

> The fact that there are three (or
> four depending if you count Linz V4) different Oberon System
> implementations, and several different compilers, and even four or
> five separate dialects of Oberon with none of them appearing to be
> really "official",

[...]

The fortunate thing about different Python implementations and in
contrast to a lot of other languages and their implementations is that
the most actively developed Python implementation, CPython, is very
portable and is present on all major operating systems of consequence
(capable of running it). Other languages have suffered because there'd
be a UNIX version and then a version for Windows or the Mac which
wasn't as well maintained, or perhaps no version at all for one or
more of these platforms. So with Python, even if one implementation is
lagging behind (eg. Jython) there's still likely to be some deployment
solution on one's operating system of choice, given some level of
flexibility.

Where incompatibilities may arise with Python implementations is in
the level of support for recent language developments and additions.
Again, CPython's portability prevents this from becoming an operating
system issue, but we see rifts between implementations targeting
different runtime platforms, and whilst CPython 2.x and CPython 3.x
may continue to cover similar operating systems (although a desire to
drop support for various operating systems was stated), it's the rift
between them that presents the risk to any common notion of what
Python actually is.

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: f---ing typechecking

[Delaney, Timothy (Tim)]

Nick Craig-Wood wrote:

> x += a
> 
> does not equal
> 
> x = x + a
> 
> which it really should for all types of x and a

Actually, this will *never* be the case for classes that do in-place
augmented assignment.

a = [1]
b = [2]

c = a + b
print a, b, c

a += b
print a, b, c

You'll note that I didn't rebind 'a' in the non-augmented assignment. If
you do, augmented and non-augmented assignment may look the same, but
they can be very different.

Tim Delaney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: file io (lagged values) newbie question

[Steven D'Aprano]

On Tue, 20 Feb 2007 22:03:43 +0100, Bruno Desthuilliers wrote:

[snip]

>>>And that's not cruft?
>> 
>> 
>> No. Why do you think it is crufty?
> 
> Because it is ?

*shrug* 

"Is not."
"Is too."
"Is not."

Why is converting a list of integers to a string all at once more crufty
than converting them one at a time? Because you have to remove the
delimiters? That's no bigger a deal than adding spaces or newlines, and
if removing the commas worries you, change the output format to separate
the numbers with comma instead of space. 



>> Would it be less crufty if I wrote it as a cryptic one liner without
>> comments?
>> 
>> f.write(str(data)[1:-1].replace(',', '') + '\n')
> 
> Nope. It's still a WTF.
> 
>> Okay, it depends on the string conversion of a list.
> 
> Nope. It depends on the *representation* of a list.

No, that would be repr(data) instead of str(data). An insignificant detail
for lists, but potentially very different for other data types.


>> But that's not going
>> to change any time soon.
> 
>> 
>>>Try this: f.write(' '.join(str(x) for x in data) + '\n')
>> 
>> 
>> That will only work in Python 2.5 or better.
> 
> Really ?

[snip demonstration]

Well, I'll be hornswaggled. Who snuck generator expressions into 2.4? I
thought they were only in 2.5 and up.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

setup.py installation and module search path

[Russ]

When I run setup.py to install a pure python package, is it supposed
to
automatically set my search path to find the installed modules? Or am
I
supposed to set my PYTHONPATH variable myself in my .bashrc file?

And what if I don't have root priviledge? Then what is supposed to
happen? Can anyone give me a clue? Thanks.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Regex Speed

[John Machin]

On Feb 21, 8:29 am, [EMAIL PROTECTED] wrote:
> While creating a log parser for fairly large logs, we have run into an
> issue where the time to process was relatively unacceptable (upwards
> of 5 minutes for 1-2 million lines of logs). In contrast, using the
> Linux tool grep would complete the same search in a matter of seconds.
>
> The search we used was a regex of 6 elements "or"ed together, with an
> exclusionary set of ~3 elements.

What is an "exclusionary set"? It would help enormously if you were to
tell us what the regex actually is. Feel free to obfuscate any
proprietary constant strings, of course.

> Due to the size of the files, we
> decided to run these line by line,

I presume you mean you didn't read the whole file into memory;
correct? 2 million lines doesn't sound like much to me; what is the
average line length and what is the spec for the machine you are
running it on?

> and due to the need of regex
> expressions, we could not use more traditional string find methods.
>
> We did pre-compile the regular expressions, and attempted tricks such
> as map to remove as much overhead as possible.

map is a built-in function, not a trick. What "tricks"?

>
> With the known limitations of not being able to slurp the entire log
> file into memory, and the need to use regular expressions, do you have
> an ideas on how we might speed this up without resorting to system
> calls (our current "solution")?

What system calls? Do you mean running grep as a subprocess?

To help you, we need either (a) basic information or (b) crystal
balls. Is it possible for you to copy & paste your code into a web
browser or e-mail/news client? Telling us which version of Python you
are running might be a good idea too.

Cheers,
John

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: setup.py installation and module search path

[Larry Bates]

Russ wrote:
> When I run setup.py to install a pure python package, is it supposed
> to
> automatically set my search path to find the installed modules? Or am
> I
> supposed to set my PYTHONPATH variable myself in my .bashrc file?
> 
> And what if I don't have root priviledge? Then what is supposed to
> happen? Can anyone give me a clue? Thanks.
> 
I'm no expert, but I think what normally happens is the module gets
installed into ../pythonxx/lib/site-packages/ and if it
installs __init__.py file there they get automatically searched.
At least that the way things work for me.

-Larry
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: setup.py installation and module search path

[Russ]

Larry Bates wrote:

> I'm no expert, but I think what normally happens is the module gets
> installed into ../pythonxx/lib/site-packages/ and if it
> installs __init__.py file there they get automatically searched.
> At least that the way things work for me.

But if I don't have root priviledge, that doesn't happen. Is there a
setup.py option to get a
package installed just in my own account in such a way that my module
search path gets
updated?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do a Decorator Here?

[Matimus]

On Feb 20, 12:20 pm, "Gregory Piñero" <[EMAIL PROTECTED]> wrote:
> Need some decorator help.
>
> I have a class.  And I want to add behavior to one of this class's
> methods to be run before the class runs the actual method.  Is this
> what decorators are for?
>
> So the class I want to work with is string.Template
>
> Let's say I have this:
> from string import Template
> a=Template("$var1 is a test")
>
> def preprocess(var1):
>#Real code here will be more complicated, just an example
> var1=var1.upper()
>
> a.substitute(var1="greg")
>
> So how can I have preprocess run before substitute is run?  I want the
> user to be able to call a.substitute and have preprocess run
> automatically.
>
> Or is this not what decorators do?  I'm trying to avoid subclassing if I can.
>
> Thanks in advance for the help.
>
> -Greg

You could just overload Template and put your own version of
substitute in there. I'm not sure a decorator is appropriate in this
case. Here is an overloading example:

[code]
from string import Template

# this assumes you are only interested in keyword arguments
# the form substitute(map) will not work
class MyTemplate(Template):
   def substitute(self,**kwargs):
  for k,v in kwargs.iteritems():
  kwargs[k] = v.upper()
  return super(MyTemplate,self).substitute(self,**kwargs)

if __name__ == "__main__":
a = MyTemplate("$var1 is a test")
print a.substitute(var1 = "greg")
[/code]

-- 
http://mail.python.org/mailman/listinfo/python-list

converting u'11\xa022' to '11\xa022'

[alf]

Hi,
is there a more elegant way to do that:

''.join([chr(ord(i)) for i in u'11\xa022' ])

-- 
thx, alf
-- 
http://mail.python.org/mailman/listinfo/python-list