Re: How to show percentage

[George Sakkis]

"Sen-Lung Chen" <[EMAIL PROTECTED]> wrote:
> Dear All:
>  I have a question of show percentage.
> For example ,I want to show the percentage of 1/3 = 33.33%
>
>  I use the 1*100/3 = 33
> it is 33 not 33.33 , how to show the 33.33 %
>  Thanks

"%.2f%%" % (100./3.)

Not quite the most readable expression in python.. ;)

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie regular expression and whitespace question

[George Sakkis]

"googleboy" <[EMAIL PROTECTED]> wrote:
> Hi.
>
> I am trying to collapse an html table into a single line.  Basically,
> anytime I see ">" & "<" with nothing but whitespace between them,  I'd
> like to remove all the whitespace, including newlines. I've read the
> how-to and I have tried a bunch of things,  but nothing seems to work
> for me:
>
> [snip]

As others have shown you already, you need to use the sub method of the re 
module:

import re
regex = re.compile(r'>\s*<')
print regex.sub('><',data)

> For extra kudos (and I confess I have been so stuck on the above
> problem I haven't put much thought into how to do this one) I'd like to
> be able to measure the number of characters between the  & 
> tags, and then insert a newline character at the end of the next word
> after an arbitrary number of characters.   I am reading in to a
> script a bunch of paragraphs formatted for a webpage, but they're all
> on one big long line and I would like to split them for readability.

What I guess you want to do is wrap some text. Do not reinvent the wheel, 
there's already a module
for that:

import textwrap
print textwrap.fill(oneBigLongLine, 60)

HTH,
George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using distutils 2.4 for python 2.3

[George Sakkis]

"Noam Raphael" <[EMAIL PROTECTED]> wrote:

> Hello,
>
> I want to distribute a package. It's compatible with Python 2.3.
> Is there a way to use distutils 2.4 feature package_data, while
> maintaining the distribution compatible with python 2.3 ?
>
> Thanks,
> Noam Raphael

You could distribute the whole 2.4 distutils package in yours so that setup.py 
finds it before the
default one. If distutils 2.4 is compatible with 2.3, that should work.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Most direct way to strip unoprintable characters out of a string?

[George Sakkis]

"Steve Bergman" <[EMAIL PROTECTED]> wrote:

> When sanitizing data coming in from HTML forms, I'm doing this (lifted
> from the Python Cookbook):
>
> from string import maketrans, translate, printable
> allchars = maketrans('','')
> delchars = translate(allchars, allchars, printable)
> input_string = translate(input_string, allchars, delchars)
>
> Which is OK.  But it seems like there should be more straightforward way
> that I just haven't figured out.  Is there?

If by straightforward you mean one-liner, there is:
''.join(c for c in input_string if c not in string.printable)

If you care about performance though, string.translate is faster; as always, 
the best way to decide
on a performance issue is to profile the alternatives on your data and see if 
it's worth going for
the fastest one at the expense of readability.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing an HTML a tag

[George Sakkis]

"Stephen Prinster" <[EMAIL PROTECTED]> wrote:
> George wrote:
> > How can I parse an HTML file and collect only that the A tags. I have a
> > start for the code but an unable to figure out how to finish the code.
> > HTML_parse gets the data from the URL document. Thanks for the help
>
> Have you tried using Beautiful Soup?
>
> http://www.crummy.com/software/BeautifulSoup/

I agree; you can do what you want in two lines:

from BeautifulSoup import BeautifulSoup
hrefs = [link['href'] for link in BeautifulSoup(urllib.urlopen(url)).fetch('a')]

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing an HTML a tag

[George Sakkis]

"George" <[EMAIL PROTECTED]> wrote:

> I'm very new to python and I have tried to read the tutorials but I am
> unable to understand exactly how I must do this problem.
>
> Specifically, the showIPnums function takes a URL as input, calls the
> read_page(url) function to obtain the entire page for that URL, and
> then lists, in sorted order, the IP addresses implied in the " HREF=· · ·>" tags within that page.
>
>
> """
> Module to print IP addresses of tags in web file containing HTML
>
> >>> showIPnums('http://22c118.cs.uiowa.edu/uploads/easy.html')
> ['0.0.0.0', '128.255.44.134', '128.255.45.54']
>
> >>> showIPnums('http://22c118.cs.uiowa.edu/uploads/pytorg.html')
> ['0.0.0.0', '128.255.135.49', '128.255.244.57', '128.255.30.11',
> '128.255.34.132', '128.255.44.51', '128.255.45.53',
> '128.255.45.54', '129.255.241.42', '64.202.167.129']
>
> """
>
> def read_page(url):
>  import formatter
>  import htmllib
>  import urllib
>
>  htmlp = htmllib.HTMLParser(formatter.NullFormatter())
>  htmlp.feed(urllib.urlopen(url).read())
>  htmlp.close()
>
> def showIPnums(URL):
>  page=read_page(URL)
>
> if __name__ == '__main__':
>  import doctest, sys
>  doctest.testmod(sys.modules[__name__])


You forgot to mention that you don't want duplicates in the result. Here's a 
function that passes
the doctest:

from urllib import urlopen
from urlparse import urlsplit
from socket import gethostbyname
from BeautifulSoup import BeautifulSoup

def showIPnums(url):
"""Return the unique IPs found in the anchors of the webpage at the given
url.

>>> showIPnums('http://22c118.cs.uiowa.edu/uploads/easy.html')
['0.0.0.0', '128.255.44.134', '128.255.45.54']
>>> showIPnums('http://22c118.cs.uiowa.edu/uploads/pytorg.html')
['0.0.0.0', '128.255.135.49', '128.255.244.57', '128.255.30.11', 
'128.255.34.132',
'128.255.44.51', '128.255.45.53', '128.255.45.54', '129.255.241.42', 
'64.202.167.129']
"""
hrefs = set()
for link in BeautifulSoup(urlopen(url)).fetch('a'):
try: hrefs.add(gethostbyname(urlsplit(link["href"])[1]))
except: pass
return sorted(hrefs)


HTH,
George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Most direct way to strip unoprintable characters out of a string?

[George Sakkis]

"Steve Bergman" <[EMAIL PROTECTED]> wrote:

> George Sakkis wrote:
>
> >
> >
> >If by straightforward you mean one-liner, there is:
> >''.join(c for c in input_string if c not in string.printable)
> >
> >If you care about performance though, string.translate is faster; as always, 
> >the best way to
decide
> >on a performance issue is to profile the alternatives on your data and see 
> >if it's worth going
for
> >the fastest one at the expense of readability.
> >
> >
> >
> Thank you for the reply.  I was really thinking of some function in the
> standard library like:
>
> s = stripUnprintable(s)
>
> When I learned php, I more or less took the route of using whatever I
> found that 'worked'.  In learning Python, I'm trying to take my time and
> learn the 'right' (that's pronounced 'beautiful') way of doing things.
>
> As it stands, I've stashed the string.translate code in a short function
> with a comment explaining what it does and how.  I mainly didn't want to
> use that if there was some trivial built-in that everyone else uses.

No there's not a stripUnprintable in a standard module AFAIK, and that's a good 
thing; if every
little function that one might ever wanted made it to the standard library, the 
language would be
overwhelming.

Make sure you calculate the unprintable characters only the first time it is 
called, not every time.
Here's a way to encapsulate this in the same function, without polluting the 
global namespace with
allchars and delchars:

import string

def stripUnprintable(input_string):
try: filterUnprintable = stripUnprintable.filter
except AttributeError: # only the first time it is called
allchars = string.maketrans('','')
delchars = allchars.translate(allchars, string.printable)
filterUnprintable = stripUnprintable.filter = lambda input: 
input.translate(allchars,
delchars)
return filterUnprintable(input_string)

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unittest setup

[George Sakkis]

"paul kölle" <[EMAIL PROTECTED]> wrote:
>
> [snipped]
>
> It seems to me my case is not that exotic, I thought it would be quite
> natural to write the boilerplate stuff in setUp() and build on that to
> step through the applications state with test* methods each building on
> top of each other. Is that the wrong approach? Are there other
> frameworks supporting such a style?

Yes, py.test: http://codespeak.net/py/current/doc/test.html.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Struggling with basics

[George Sakkis]

"Jason" <[EMAIL PROTECTED]> wrote:

> What I'd like to know is do you think it would be better to sort the
> list in memory, or print it out sorted?  If the latter, then naturally
> I'd need to change the showScores section to show the list in a reverse
> order.  But, would sorting the list in memory be more effective?

The list *is* sorted; the thing is that it is in ascending order (from lowest 
to highest) but you
would rather have it in descending. There are (at least) two alternatives:

1. Keep the list as it is now in ascending order and print it in reverse. In 
python 2.4, this is as
elegant and efficient as it can, using the reversed() builtin function. Just 
replace in showScores
"for score,name in self.hiScores" with "for score,name in 
reversed(self.hiScores)". reversed()
returns an iterator over the sequence, not a new list, so the memory overhead 
is minimal.

2. Instead of storing (score,name) pairs, store (-score,name). When a list of 
the latter is in
ascending order, the former is in descending. In this case of course, you have 
to make sure that
showScores() and lastScore() return the actual (positive) score, not the stored 
(negative) one.

I would go for the first alternative but YMMV.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Struggling with this concept please help

[George Sakkis]

"George" <[EMAIL PROTECTED]> wrote:

> Hello everyone I know many have helped but I cannot get this to work
> out correctly. I cannot use BeautifulSoup at all. I need to:
> [snipped]

What do you mean you cannot use BeautifulSoup ? You cannot download it, install 
it, import it, or
you are not allowed to use it because it's a homework ? If it's the latter, I 
doubt that you'll get
a solution spelled out for you in this group.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 350: Codetags

[George Sakkis]

"Paul Rubin"  wrote:

> I'm opposed to pretty much every proposal of this sort.  If you want
> to propose adding a feature to the language, add it in a way that the
> compiler can know about it and notice when it's not used correctly.
> Mere conventions that are not checked by the compiler are just more
> stuff for people to remember.  That doesn't say they're always useless
> but in general they should not be the subject of PEP's.

There are more than a dozen "informational PEPs" so far, or two dozens if you 
count the meta-PEPs as
well (http://www.python.org/peps/). This PEP states upfront that it is 
informational, so I don't see
the problem, unless are you suggesting a (meta-)PEP against informational PEPs 
in general :-)

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Sparse matrices

[George Sakkis]

Is there any sparse matrix package compatible with Numeric/Numarray ? Ideally, 
the implementation of
a matrix (dense/sparse) should be transparent to the application. However the 
APIs of the only
packages I'm aware of -- the Sparse module in Scipy and PySparse --are both 
pretty incomplete
compared to (dense) numeric arrays. Any alternatives ?

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: backreferences

[George Sakkis]

"Amy Dillavou" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
> Can someone help me with understanding how python uses backreferences?
> I need to remember the item that was last matched by the re engine but i
> cant seem to understand anything that I find on backreferences.  if I
> want to access the last match do i use \number or is there something
> else i have to do?
>
> heres part of my code:
> renDate = re.compile("$((\d){4}-(\d){2}-(\d){2}))")
> renDate.search(l)
> if(exist.search(l) and str(lastmodDate) < \1): #i need help here with \1
>
> Thanks in advance
> A.D

renDate = re.compile(some_regex)
match = renDate.search(input)
if match and str(lastmodDate) < match.group(1):
do_something()


HTH,
George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: grouping array

[George Sakkis]

<[EMAIL PROTECTED]> wrote:

> hi if I have an array
>
> say x = [[2,2,0,0,1,1],
>  [1,1,0,0,1,1],
>  [1,1,0,0,1,1]]
> I basically want to group regions that are non zero like I want to get
> the coordinates of non zero regions..as (x1,y1,x2,y2)
> [(0,0,2,1),(0,4,2,5)] which show the top left(x1,y1) and bottom
> right(x2,y2) corners of each group.hope i am clear.
>
> Thanks

I guess you imply rectangular regions only ? If not, what's the output supposed 
to be for

[[2,2,0,0,1,1],
 [1,1,3,0,0,1],
 [1,1,3,0,1,1]]

or

[[2,2,2,2],
 [1,0,3,3],
 [1,1,3,0]]

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Opinion on Pyrex

[George Sakkis]

"Carl" <[EMAIL PROTECTED]> wrote:

> I have recently started to use Pyrex and am amazed by it's useability.
>
> Are there any alternatives to Pyrex?
>
> One thing that I haven't figured out is how to embed pure C/C++ source code
> into Pyrex. For example, if you have a bunch of C files that you want to
> use together with some Python code snippets, how do you practically achieve
> this using Pyrex? I have come to the conclusion that it is not possible
> without some rewriting and adaptation (translation) of available C source
> code (if you don't want to compile and link all your C source into a
> statical or dynamical library).
>
> Carl

You may want to check out weave: http://www.scipy.org/documentation/weave/

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "no variable or argument declarations are necessary."

[George Sakkis]

"Michael" <[EMAIL PROTECTED]> wrote:

> James A. Donald wrote:
> > On Sun, 02 Oct 2005 17:11:13 -0400, Jean-Francois Doyon
> > James A. Donald:
> >>  > Surely that means that if I misspell a variable name, my program will
> >>  > mysteriously fail to work with no error message.
> >> No, the error message will be pretty clear actually :)
> > Now why, I wonder,  does this loop never end :-)
> > egold = 0
> > while egold < 10:
> >ego1d = egold+1
>
> I know (hope! :-) that's a tongue-in-cheek question, however the answer as
> to why that's not a problem is more to do with development habits rather
> than language enforcement. (yes with bad habits that can and will happen)
>
> [snipped description of test-driven development culture]

As an aside, more to the point of the specific erroneous example is the lack of 
the standard python
idiom for iteration:

for egold in xrange(10):
pass

Learning and using standard idioms is an essential part of learning a language; 
python is no
exception to this.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "no variable or argument declarations are necessary."

[George Sakkis]

"Steven D'Aprano" <[EMAIL PROTECTED]> wrote

> [snipped]
> No compiler will catch this error:
>
> x = 12.0 # feet
> # three pages of code
> y = 15.0 # metres
> # three more pages of code
> distance = x + y
> if distance < 27:
> fire_retro_rockets()

Actually modern compilers can 
(http://www.boost.org/libs/mpl/doc/tutorial/dimensional-analysis.html)
at the expense of the programmer's eye health...

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Merging sorted lists/iterators/generators into one stream of values...

[George Sakkis]

"Lasse Vågsæther Karlsen" <[EMAIL PROTECTED]> wrote:

> Ok, that one looks more sleak than what I came up with.

For the most efficient and elegant solution, check out Raymond Hettinger's 
reply to:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/141934

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python recipes: list mixin, improved timeit, etc

[George Sakkis]

"Steven D'Aprano" <[EMAIL PROTECTED]> wrote:

> I've just found this:
>
> [quote]
> A mixin class is a parent class that is inherited from - but not as
> a means of specialization. Typically, the mixin will export services to a
> child class, but no semantics will be implied about the child "being a
> kind of" the parent.
> [end quote]
>
> from http://c2.com/cgi/wiki?MixIn
>
> Is that all they are?
>
> It is amazing how you can take the simplest concept, and by using
> appropriate terminology, make it as confusing and opaque as you want...
>
> *wink*

 Now this reminds me of Xah's entertaining posts about "moronicities of the 
tech-geek industry
jargon" .

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Merging sorted lists/iterators/generators into one stream of values...

[George Sakkis]

"Lasse Vågsæther Karlsen" <[EMAIL PROTECTED]> wrote:

> Thanks, that looks like Mike's solution except that it uses the
> built-in heapq module.

This make a big difference for the algorithmic complexity; replacing an item in 
a heap is much more
efficient than sorting the whole list.

> While this one contains less code than Mike's solution it seems to lack
> the ability to control the comparison operation, which means it won't
> work in my case. I need to both be able to sort on an arbitrary field
> name (could be done using a list where I place the field first), and
> also to be able to sort in a different order than smallest-first.
>
> Perhaps by adjusting the data that is returned through each source
> would do that. I'll look into it.

Yes, it's a little inconvenient that the builtin heap doesn't take a comparison 
operation but you
can easily roll your own heap by transforming each item to a (key,item) tuple. 
Now that I'm thinking
about it, it might be a good addition to the cookbook.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Objects with different data views

[George Sakkis]

"Steven D'Aprano" <[EMAIL PROTECTED]> wrote:

> I'm not sure how to do this, or where to start looking for the right
> information, so any advice would be appreciated.
>
> I want to implement a class with two (or more) different ways of looking
> at its attributes.
>
> One example of this might be complex numbers, which can be written in
> Cartesian form (x+yi) or polar form (r cis theta).
>
> (Yes, I know Python already has complex numbers. That was just an example.)
>
> Another might be 3D vectors, which can be written in Cartesian form
> [x, y, z], polar form [r, theta, z] or spherical polar [r, theta, phi].
>
> It is important that there are no privileged attributes, e.g. in the
> above example, I can set any of x, y, z, r, theta or phi and all the
> others will automatically reflect the changes. A concrete, if simple,
> example will make it clear.
>
> Suppose I have a transformation (a,b) <-> (x,y) where:
>
> x = a+b
> y = a+2*b
>
> I create an instance spam, and set a and b:
>
> spam.a = 1
> spam.b = 2
>
> Now I should be able to read x and y:
>
> print spam.x, spam.y
> # prints 3 5
>
> If I set attribute y:
>
> spam.y = 0
>
> a and b automatically change to match:
>
> print spam.a, spam.b
> # prints 6, -3
>
>
> Anyone have any good ideas for how I should implement this?

As others have replied, properties is the way to go. There have been a few 
recipes in the Cookbook
that avoid cluttering the class namespace with temporary get/set/del methods, 
e.g.
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410698.

HTH,
George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best way to share a python list of objects

[George Sakkis]

"kyle.tk" <[EMAIL PROTECTED]> wrote:

> So I have a central list of python objects that I want to be able to
> share between different process that are possibly on different
> computers on the network. Some of the processes will add objects to
> list and another process will be a GUI that will view objects in the
> list. I want this all to happen in real-time (e.g once a processes adds
> an object to the list the GUI will see it.)
>
> What would be the best way to accomplish this. Some of my ideas:
> - An XML file r/w-able by all processes
> - Send pickled objects between all processes and each keeps it own list
> locally
> - A ascii type protocol akin to ftp the hands out all the info to the
> processes
>
> Any other ideas? What would work the best

If all the processes are python, I would check Pyro first: 
http://pyro.sourceforge.net/.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Merging sorted lists/iterators/generators into one stream of values...

[George Sakkis]

"Lasse Vågsæther Karlsen" <[EMAIL PROTECTED]> wrote:

> Alex Martelli wrote:
> > George Sakkis <[EMAIL PROTECTED]> wrote:
> 
> >>Yes, it's a little inconvenient that the builtin heap doesn't take a
> >>comparison operation but you can easily roll your own heap by transforming
> >>each item to a (key,item) tuple. Now that I'm thinking about it, it might
> >>be a good addition to the cookbook.
> >
> >
> > I believe Python 2.5 adds a key= argument to heapq's functions...
> 
>
> I will revisit the heapq solution when 2.5 is released then.
>
> Thanks for the heads up. For the moment I will stay with the list
> solution that Mike came up with slightly changed to accomodate tips and
> pointers from others in this thread.

Just added a recipe at 
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440673. You can try
both and see if there's any significant performance difference for your data.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function decorator that caches function results

[George Sakkis]

"Lasse Vågsæther Karlsen" <[EMAIL PROTECTED]> wrote:

> [snip]
>
> Ok, so I thought, how about creating a decorator that caches the
> function results and retrieves them from cache if possible, otherwise it
> calls the function and store the value in the cache for the next invokation.
>
> [snip]

Cool, you re-invented the memoization pattern:
http://en.wikipedia.org/wiki/Memoization
http://aspn.activestate.com/ASPN/search?query=memoize&x=0&y=0§ion=PYTHONCKBK&type=Subsection

Yes, it's kinda discouraging that most interesting ideas have already been 
conceived, implemented
and used by others...

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Merging sorted lists/iterators/generators into one stream of values...

[George Sakkis]

"Lasse Vågsæther Karlsen" <[EMAIL PROTECTED]> wrote:

> George Sakkis wrote:
> 
> > Just added a recipe at 
> > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440673. You can
try
> > both and see if there's any significant performance difference for your 
> > data.
> 
>
> Thanks, will take a look at it later. The sort solution seems to work
> nicely. Might be a big difference if I have a lot of sources though as I
> bet the overhead in doing a sort of N items gets higher than doing a
> manipulation of a heap to place an item in the right spot, but with 4-5
> or a few more sources might not make an impact at all.

Unless you're talking about hundreds or thousands sources, it probably
won't. I would still go for the heap solution since IMO the resulting
code it's more readable and easier to understand.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how do you pronounce wxpython

[George Sakkis]

"Alex" <[EMAIL PROTECTED]> wrote:

> My native language is not English so I just wonder how you pronounce
> wxPython.
>
> vi-ex python
> double-you-ex python
> wax-python
>
> or something else
>
> Thanks

I am sure it is pronounced the same way as wxWidgets .

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python reliability

[George Sakkis]

Steven D'Aprano wrote:

> On Sun, 09 Oct 2005 23:00:04 +0300, Ville Voipio wrote:
>
> > I would need to make some high-reliability software
> > running on Linux in an embedded system. Performance
> > (or lack of it) is not an issue, reliability is.
>
> [snip]
>
> > The software should be running continously for
> > practically forever (at least a year without a reboot).
> > Is the Python interpreter (on Linux) stable and
> > leak-free enough to achieve this?
>
> If performance is really not such an issue, would it really matter if you
> periodically restarted Python? Starting Python takes a tiny amount of time:

You must have missed or misinterpreted the "The software should be
running continously for practically forever" part. The problem of
restarting python is not the 200 msec lost but putting at stake
reliability (e.g. for health monitoring devices, avionics, nuclear
reactor controllers, etc.) and robustness (e.g. a computation that
takes weeks of cpu time to complete is interrupted without the
possibility to restart from the point it stopped).

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie Count Question

[George Sakkis]

"ProvoWallis" wrote:

> I've managed to get this far thanks to looking at other
> posts on the board but no matter waht I try all of the
> sections end up being numbered for the total number of
> sections in the document. e.g., if there are 100 sections
> in the document the "no" attribute is "1.100"
> for each one.

Of course it is; the counter you compute is fixed and equal to
len(all). What you need is a substitution function that keeps track of
the counter and increments it by one for every substitution. This means
that the counter becomes part of the function's state. When you hear
"function" and "state" together, the typical solution is "class":

import re

class ReplaceSecMain(object):
def __init__(self):
self._count = 0
self._secmain_re = re.compile(r'''
(?<=  ) # positive lookahead assertion
''', re.IGNORECASE | re.VERBOSE)

def sub(self, text):
return self._secmain_re.sub(self._subNum, text)

def _subNum(self, match):
self._count += 1
return '%s.%.2d' % (match.group(1), self._count)


print ReplaceSecMain().sub(open("myfile.txt").read())


I also cleaned up the regular expression a little; google for
lookahead/lookbehind assertions if you haven't seen them before.

HTH,
George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Merging sorted lists/iterators/generators into one stream of values...

[George Sakkis]

"Alex Martelli" <[EMAIL PROTECTED]> wrote:

> George Sakkis <[EMAIL PROTECTED]> wrote:
>...
> > > manipulation of a heap to place an item in the right spot, but with 4-5
> > > or a few more sources might not make an impact at all.
> >
> > Unless you're talking about hundreds or thousands sources, it probably
> > won't. I would still go for the heap solution since IMO the resulting
> > code it's more readable and easier to understand.
>
> I'm not so sure about either sentence...:
>
> Helen:~/pynut/samp alex$ python merger.py --numstreams=10 --minlen=100
> --how=S
> Best time for 10 loops: 0.247116088867
>
> Helen:~/pynut/samp alex$ python merger.py --numstreams=10 --minlen=100
> --how=H
> Best time for 10 loops: 0.10344004631
>
> i.e., a heap solution may be over 4 times faster than a sort-based one
> (in the following implementations).

Interesting; I thought timsort on small almost ordered lists would be 
practically as fast as the
heap. Still, how is 0.10344 over 4 times faster than 0.247116 ?

> Readability seems quite comparable
> (skipping the rest of the infrastructure, which generates random sorted
> streams and ensures a stream is exhausted and verifies it etc etc):
>
> def merge_by_sort(streams):
>   sources = [[s.next(), i, s.next] for i, s in enumerate(streams)]
>   while sources:
> sources.sort(reverse=True)
> best_source = sources[-1]
> yield best_source[0]
> try: best_source[0] = best_source[-1]()
> except StopIteration: sources.pop()
>
> def merge_by_heap(streams):
>   sources = [[s.next(), i, s.next] for i, s in enumerate(streams)]
>   heapq.heapify(sources)
>   while sources:
> best_source = sources[0]
> yield best_source[0]
> try: best_source[0] = best_source[-1]()
> except StopIteration: heapq.heappop(sources)
> else: heapq.heapreplace(sources, best_source)

Indeed, these are almost equivalent as far as readability goes; the previous 
examples in the thread
were less clear. By the way, why do you need 'i' and enumerate above ?

> Hmmm, I wonder if something like merge_by_heap would be a good candidate
> for itertool.  Raymond...?
>
>
> Alex

Yes, it would be nice to make it into 2.5.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: best Pythonic way to do this sort: Python newb

[George Sakkis]

"Satchidanand Haridas" <[EMAIL PROTECTED]> wrote:

> >So, I want to sort myList by the return of myFunction( value3 )
> >
> >I tried doing the following... with no luck so far
> >myList.sort(lambda x, y: cmp(myFunction(x[2]), myFunction(y[2]))
> >
> >
> >
> I think the above statement should be as follows:
>
> myList.sort(lambda x, y: cmp(myFunction(x[2]) - myFunction(y[2]))
>
>
>
> hope that helps.

It would help more if you tested it before you posted. cmp takes two arguments 
(let alone that
subtraction may not be defined for the list elements), so the original version 
is correct.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparing lists

[George Sakkis]

"Christian Stapfer" <[EMAIL PROTECTED]> wrote:

> <[EMAIL PROTECTED]> wrote:
> > try to use set.
>
> Sorting the two lists and then extracting
> A-B, B-A, A|B, A & B and A ^ B in one single
> pass seems to me very likely to be much faster
> for large lists.

Why don't you implement it, test it and time it to be more convincing about 
your intuition ?

George


-- 
http://mail.python.org/mailman/listinfo/python-list

set.__getstate__ not overriden

[George Sakkis]

>>> import pickle
>>> class Foo(set):
... def __getstate__(self): assert False

>>> pickle.dumps(Foo())
# doesn't raise AssertionError

The same happens with frozenset. Before I submit it to the bugs
tracker, is there a chance this is a 'feature' ?

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Info] PEP 308 accepted - new conditional expressions

[George Sakkis]

"Dave Hansen" <[EMAIL PROTECTED]> wrote:

> On Mon, 10 Oct 2005 16:42:34 -0500, Terry Hancock
> <[EMAIL PROTECTED]> wrote:
>
> >On Sunday 09 October 2005 07:50 am, phil hunt wrote:
> >> On Fri, 7 Oct 2005 01:05:12 -0500, Terry Hancock <[EMAIL PROTECTED]> wrote:
> >> >GvR's syntax has the advantage of making grammatical sense in English 
> >> >(i.e.
> >> >reading it as written pretty much makes sense).
> >>
> >> I know, let's re-write Python to make it more like COBOL! That's
> >> bound to be a winner!
> >
> >Whereas the "natural order" of "condition affirmative negative" is natural
> >for what reason?  That it is so in C?
>
> And Basic, and Fortran, and Lisp, and just about any programming
> language you care to name, including python (if Condition: Affirmative
> else: Negative).

Block delimiters (curly braces, if/fi, begin/end, etc.) are also in just about 
any language but this
didn't stop python using indentation instead, so what's your point ? Conformity 
and backwards
compatibility should not be top priorities in language design; fortunately for 
python, they're not.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to do *args, **kwargs properly

[George Sakkis]

"Lasse Vågsæther Karlsen" <[EMAIL PROTECTED]> wrote:

> So what you're saying is that instead of:
>
> def fn(*values, **options):
>
> I should use:
>
> def fn(values, cmp=cmp):
>
> in this specific case?
>
> and then instead of:
>
> fn(1, 2, 3, cmp=...)
>
> this:
>
> fn([1, 2, 3], cmp=...)
>
> 
>
> I think I'll re-write to use a list instead

Actually in most cases you don't need to assume it's a list; any
iterable is usually good enough. You can always turn it into a list (or
a tuple or a set or..) in the function if you really have to. So when
you do have to and when you don't ? You don't have to if all you do is
iterate over the elements. This is true even if you want to iterate
more than once; just use the itertools.tee() function to create N
independent iterators. To sum up, a better signature for your function
is likely to be "def fn(iterable, cmp=cmp)".

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Merging sorted lists/iterators/generators into one stream of values...

[George Sakkis]

> Function name is perhaps not the best one. It occurs to me that this
> is the GROUP BY in SQL so perhaps a different name is better, but
> then again this might be moot if such a already exists somewhere :)

Amazing, you keep reinventing things, even with the exact same name :)

from itertools import imap,groupby
from operator import itemgetter

for fruit,group in groupby(fruits, itemgetter(0)):
print fruit, "has a sum of", sum(imap(itemgetter(1),group))

For this to work as intended, fruits has to be already sorted by the
same key given to grouby; otherwise just replace fruits with
sorted(fruits, itemgetter(0)).

By the way, read all the functions in the itertools module
(http://docs.python.org/lib/itertools-functions.html), it will save you
a lot of time.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A Tree class, my $0.02 contribution to the python community.

[George Sakkis]

"Antoon Pardon" <[EMAIL PROTECTED]> wrote:
> Comments are welcome:
>
>   http://www.pardon-sleeuwaegen.be/antoon/avltree.html

How about adding two shortcut methods, nextkey(k) and prevkey(k), to return the 
next and previous
key respectively ? For instance nextkey would be equivalent to (untested):

def nextkey(self, key):
iter = self[key:]
first = iter.next()
if key not in self: return first
else: return iter.next()

Also for consistency, nextvalue(k), prevvalue(k), nextitem(k), previtem(k) 
would be reasonable
additions.

And a question: what does step do if the keys are not integers since you 
restrict step to be integer
?

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiple assignments simplification

[George Sakkis]

<[EMAIL PROTECTED]> wrote:

> [snipped]
>
> Do you know some algorithm (or you can give some suggestions) to
> minimize the number of simple assignments needed for a "regular"
> situation like that?

You can formulate the task as a graph-theoretic problem by representing the set 
of assignments as a
digraph G(V,E), where:
- V = set(LHS) | set(RHS): the vertex set V is the union of all left and right 
hand side
expressions.
- E = set((v1,v2) for "v1 = v2" in assignments): there is an edge from v1 to v2 
for every assignment
v1=v2.

Now, every edge v1->v2 where in-degree(v1)==0 corresponds to a safe assignment: 
v1 is not assigned
to any RHS, so it can be overwritten. After the assignment, both v1 and the 
edge (v1,v2) can be
removed, decrementing the in-degree of v2. This happens iteratively as long as 
there are nodes with
zero in-degree.

At this point, all remaining nodes have in-degree >=1 and they form one or more 
strongly connected
components. Since no RHS occurs more than once*, the out-degree of every vertex 
is less than or
equal to 1. Therefore, for every component Ci,
|Vi| >= sum(out-degree(v) for v in Vi)  == |Ei|.

Since for a strongly connected component |Vi| <= |Ei|, the previous 
relationship is actually
equality |Vi| == |Ei|. Thus each component is a simple cycle 
v[1]->v[2]->...v[n]->v[1]. You can
break the cycle by introducing an auxiliary variable x in an arbitrary edge, 
say v[n]->v[1]. Then
the following assignments can take place: x = v[1]; v[1] = v[2]; v[2] = v[3]; 
...; v[n-1] = v[n];
v[n] = x

So overall, you need just one auxiliary variable for each strongly component of 
G.

HTH,
George


* More than one assignments with the same RHS [v=a, v=b] are useless since only 
the last has an
effect. In any case, the useless assignments can be filtered out in the 
beginning.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to make this code faster

[George Sakkis]

<[EMAIL PROTECTED]> wrote:

> hello,
>
> I found that scipy only works with python 2.3 or?

You can use Numeric instead of scipy if you need/want to:

from Numeric import arange,reshape,sin

def computeMatrix(n):
xcoor = arange(0,1,1/float(n))
ycoor = reshape(xcoor, (n,1))
return sin(xcoor*ycoor) + 8*xcoor

Note that arange() does not include the endpoint, i.e. 
arange(0,1,0.25).tolist() ==[0.0, 0.25, 0.5,
0.75].

> I don't know if the logic is correct:
> 1. loop inside loop uses a lot of resources
> 2. Numeric or Numpy can make program faster
> 3. It use kind of Array/Matrix analysis style
> 4. We have to change our algorithms so that Numeric or Numpy can help
> us, Matrix style

That's correct more or less.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: NYLUG meeting: The Python Object Model with Alex Martelli & Google(open bar and food!)

[George Sakkis]

"Ron Guerin" <[EMAIL PROTECTED]> wrote:

> The New York Linux User's Group invites you to a special presentation
> by Alex Martelli of Google, on the Python Object Model.  This
> presentation will be held at P.J. Clarke's Sidecar, rather than our
> usual location, and Google is picking up the tab for an hour and a half
> of open bar and food.  Additionally, if you're looking for a job as a
> Python developer, bring your resume.
>
> Please RSVP at http://rsvp.nylug.org to attend, as seating is limited.
>
> - Ron

What date is it ? It isn't mentioned at the web site either.

George



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Searching files in directories

[George Sakkis]

"Larry Bates" wrote:

> Not tested but should be close:
>
> import os
> import shutil
>
> files = ['file1.txt']
> source_directory  = '/tmp/source/'
> destination_directory = '/tmp/destination/'
>
> for file in files:
> src=os.path.join(source_directory, file
> dst=os.path.join(destination_directory, file
> if os.path.exists(os.path.join(source_directory, file):
> shutil.copy(src, dst)


Or more succinctly using the path module:

from path import path

files = ['file1.txt']
src_dir = path('/tmp/source/')
dest_dir = path('/tmp/destination/')

for filename in files:
srcfile = src_dir / filename
if srcfile.exists():
srcfile.copy(dest_dir)


George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: object inheritance and default values

[George Sakkis]

"Ron Adam" <[EMAIL PROTECTED]> wrote:

> I'm trying to implement simple svg style colored complex objects in
> tkinter and want to be able to inherit default values from other
> previously defined objects.
>
> I want to something roughly similar to ...
>
> class shape(object):
>  def __init__(self, **kwds):
>   # set a bunch of general defaults here.
>   self.__dict__.update(kwds)
> def draw(self, x=0, y=0, scale=1.0):
>   # draw the object
>
> hello = shape(text='hello')
> redhello = hello(color='red')
> largeredhello = redhello(size=100)
> largeredhiya = largeredhello(text='Hiya!')
> largeredhiya.draw(c, 20, 50)
>
>
> I think this will need to require __new__ or some other way to do it.
> But I'm not use how to get this kind of behavior.  Maybe the simplest
> way is to call a method.
>
> redhello = hello.makenew( color='red' )

Just name it '__call__' instead of makenew and you have the syntax sugar you 
want:

def __call__(self, **kwds):
new = self.__class__(**self.__dict__)
new.__dict__.update(kwds)
return new

Personally I would prefer an explicit method name, e.g. 'copy'; hiding the fact 
that 'shape' is a
class while the rest are instances is likely to cause more trouble than it's 
worth.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function to execute only once

[George Sakkis]

"snoe" <[EMAIL PROTECTED]> wrote:

> I've been seeing alot about decorators and closures lately and my
> initial thought was that this would be a good place to use them instead
> of wrapping it around a class. That was my initial thought :) What I
> came up with was this:

Apparently you're not the first to think of it:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/425445.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: XML dom question

[George Sakkis]

"George" <[EMAIL PROTECTED]> wrote:

> How can I do the following in python:
>
> [snipped]

In exactly the same way you could do it if you actually read the replies to the 
thread with the same
topic you posted yesterday. Duh!

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help with creating a dict from list and range

[George Sakkis]

"James Stroud" <[EMAIL PROTECTED]> wrote:

> Could be even simpler since enumerate creates tuples anyway:
>
> dct = dict(x for x in enumerate(description))
>
> James
>
> On Friday 14 October 2005 08:37, Steve Holden wrote:
> >  >>> dct = dict((x[1], x[0]) for x in enumerate(description))
> >  >>> dct
> >
> > {'second': 1, 'third': 2, 'first': 0}


"James Stroud" wrote

> Could be even simpler since enumerate creates tuples anyway:
>
> dct = dict(x for x in enumerate(description))
>
> James
>
> On Friday 14 October 2005 08:37, Steve Holden wrote:
> >  >>> dct = dict((x[1], x[0]) for x in enumerate(description))
> >  >>> dct
> >
> > {'second': 1, 'third': 2, 'first': 0}

Or even simplest :-)

dct = dict(enumerate(description))

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Microsoft Hatred FAQ

[George Sakkis]

Keith Thompson wrote:

> "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> writes:
> > Hm... What does this have to do with Perl?
> >
> > Why did you post this in comp.lang.perl.misc?
>
> He posted this in comp.lang.python, comp.lang.perl.misc,
> comp.unix.programmer, comp.lang.java.programmer, *and*
> comp.os.linux.misc because he's a troll.
>
> I wish I could say that he'll go away if we ignore him.  I can say,
> however, that ignoring him will minimize his impact.  In the past, his
> rants have led to long rambling arguments across multiple newsgroups,
> none of them relevant to any point that might be made -- which is
> probably exactly what he wants.

Exactly, Mr. Lee is a self-proclaimed troll
(http://xahlee.org/Netiquette_dir/troll.html). Actually he seems to be
more than a plain troll; the style of many of his posts is indicative
of one or more mental disorders, so he'll probably continue trolling
even if he's ignored. Nevertheless, I agree that ignoring him will
minimize his impact.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dynamic generation of doc-strings of dynamically generated classes

[George Sakkis]

"Mikael Olofsson" <[EMAIL PROTECTED]> wrote:

> Hi!
>
> I've asked Google, but have not found any useful information there.
>
> Situation: I have a base class, say
>
>  >>> class base(object):
>  ImportantClassAttribute = None
>
> Now, I want to dynamically generate subclasses of base. That's not a
> problem. However, I very much want those subclasses to have individual
> doc-strings. More precicely, I want that important class attribute to be
> reflected in the doc-string. That's the problem. The only way I've
> managed to accomplish that is something like the following.
>
>  >>> ImportantClassAttribute = 7
>  >>> docString = 'The case %s.' % (ImportantClassAttribute,)
>  >>> exec('''class new(base):
>   """%s"""
>   pass ''' % (docString,))
>  >>> new.ImportantClassAttribute = ImportantClassAttribute
>  >>> new.__doc__
> 'The case 7.'
>
> This works as intended. The subclasses do get the doc-strings I want
> them to have, and I can live with this solution. But: This solution does
> not strike me as especially beautiful or readable. My first naïve
> attempt was instead the following.
>
>  >>> class new(base):
> pass
>
>  >>> new.ImportantClassAttribute = 7
>  >>> new.__doc__ = ('The case %(ImportantClassAttribute)s.'
>% new.__dict__)
>
> Traceback (most recent call last):
>File "", line 1, in -toplevel-
>  new.__doc__ = ('The case %(ImportantClassAttribute)s.'
> TypeError: attribute '__doc__' of 'type' objects is not writable
>
> This is readable to me, but apparently not the way to go, since I'm not
> allowed to replace the doc-string like this. I've also tried a number of
> other ways, but they all stumble on similar reasons.
>
> Any ideas? Am I stuck with the clumsy exec-solution, or are there other
> ways to dynamically generate doc-strings of classes?

There's nothing specifically about doc-strings, but you can create and 
customise a whole class
dynamically:

def makeBaseSubclass(impClassAttr):
return type('new_%s' % impClassAttr,
 (base,object),
{'ImportantClassAttribute': impClassAttr,
  '__doc__': 'The case %s' % impClassAttr})

>>> new = makeBaseSubclass(7)
>>> new.ImportantClassAttribute
7
>>> new.__doc__
'The case 7'


HTH,
George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mutual module imports

[George Sakkis]

"Tony Nelson" <[EMAIL PROTECTED]> wrote:

> [snipped]
>
> I have written a Python module that uses some C functions.  I wrote the
> module in two parts, one Python, one Pyrex (C).  They need to share some
> globals.
>

> [snipped]
>
> Now the Python module imports the Pyrex module and just shoves
> references to its globals into the Pyrex module (the Pyrex module
> defines them as None).  The Pyrex module doesn't import the Python
> module anymore.  This also works, even when the Python module has a
> different name (e.g. "__main__").  I just feel dirty about it.

Well, I feel dirty every time I have to share globals .

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Intersection of lists/sets -- with a catch

[George Sakkis]

"Carl Banks" <[EMAIL PROTECTED]> wrote:

> Howabout something like this (untested):
>
> class CmpProxy(object):
> def __init__(self,obj):
> self.obj = obj
> def __eq__(self,other):
> return (self.obj.att_a == other.obj.att_b
> and self.obj.att_b == other.obj.att_b)
> def __hash__(self):
> return hash((self.obj.att_a,self.obj.att_b))
>
> set_a = set(CmpProxy(x) for x in list_a)
> set_b = set(CmpProxy(y) for y in list_b)
> overlaps = [ z.obj for z in set_a.intersection(set.b) ]

Or more generally:

class Comparable(object):
def __init__(self, obj, key):
self.obj,self._key = obj,key
def __hash__(self):
return hash(self._key(self.obj))
def __eq__(self, other):
return self._key(self.obj) == self._key(other.obj)


if __name__ == '__main__':
some_list = ["hello", "world"]
another_list = ["HeLlO", "word"]
key = str.lower
some_set = set(Comparable(x,key) for x in some_list)
another_set = set(Comparable(x,key) for x in another_list)
print "overlaps:", [x.obj for x in some_set.intersection(another_set)]
print "unique_some:", [x.obj for x in some_set.difference(another_set)]
print "unique_another:", [x.obj for x in another_set.difference(some_set)]


George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how best to split into singleton and sequence

[George Sakkis]

"Randy Bush" <[EMAIL PROTECTED]> wrote:

> >>> l = []
> >>> s = 'a|b'
> >>> t, l = s.split('|')
> >>> t
> 'a'
> >>> l
> 'b'
> >>> s = 'a|b|c|d'
> >>> t, l = s.split('|')
> Traceback (most recent call last):
>   File "", line 1, in ?
> ValueError: too many values to unpack
> >>>
>
> so, i imagine what is happening is the lhs, t,l, is really
> (t, (l)), i.e. only two items.
>
> so how should i have done this readably and simply?

>>> s = 'a|b|c|d'
>>> l = s.split('|')
>>> t = l.pop(0)

By the way, don't use 'l' as an identifier; it is very close to '1' visually.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Upper/lowercase regex matching in unicode

[George Sakkis]

"Jason Stitt" <[EMAIL PROTECTED]> wrote:

> What's the best way to match uppercase or lowercase characters with a
> regular expression in a unicode-aware way? Obviously [A-Z] and [a-z]
> aren't going to cut it. I thought there were character classes of the
> form ::upper:: or similar syntax, but can't find them in the docs.
> Maybe I'm getting it mixed up with Perl regexen.
>
> The upper() and lower() methods do work on accented characters in a
> unicode string, so there has to be some recognition of unicode case
> in there somewhere.
>
> Thanks,
>
> Jason

http://tinyurl.com/7jqgt

George


-- 
http://mail.python.org/mailman/listinfo/python-list

DBM scalability

[George Sakkis]

I'm trying to create a dbm database with around 4.5 million entries but the 
existing dbm modules
(dbhash, gdbm) don't seem to cut it. What happens is that the more entries are 
added, the more time
per new entry is required, so the complexity seems to be much worse than 
linear. Is this to be
expected and if so, should I expect better performance (i.e. linear or almost 
linear) from a real
database, e.g. sqlite ?

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: DBM scalability

[George Sakkis]

"Paul Rubin" <http://[EMAIL PROTECTED]> wrote:

> "George Sakkis" <[EMAIL PROTECTED]> writes:
> > I'm trying to create a dbm database with around 4.5 million entries
> > but the existing dbm modules (dbhash, gdbm) don't seem to cut
> > it. What happens is that the more entries are added, the more time
> > per new entry is required, so the complexity seems to be much worse
> > than linear. Is this to be expected
>
> No, not expected.  See if you're using something like db.keys() which
> tries to read all the keys from the db into memory, or anything like that.

It turns out it doesn't have to do with python or the dbm modules. The same 
program on a different
box and platform runs linearly, so I guess it has to do with the OS and/or the 
hard disk
configuration.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to separate directory list and file list?

[George Sakkis]

"Gonnasi" <[EMAIL PROTECTED]> wrote:

> With
> >glob.glob("*")
>
> or
> >os.listdir(cwd)
>
> I can get a combined file list with directory list, but I just wanna a
> bare file list, no directory list. How to get it?
>
> Tons of thanks in advance!
>
> Gonnasi

Using the path module (http://www.jorendorff.com/articles/python/path/):

from path import path
path('.').files()

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: XML Tree Discovery (script, tool, __?)

[George Sakkis]

"[EMAIL PROTECTED]" wrote:

> Hi all,
>
> Finally diving into XML programmatically.  Does anyone have a best
> practice recommendation for programmatically discovering the structure
> of an arbitrary XML document via Python?
>
> It seems like it is a common wheel I'd be re-inventing.
>
> Thanks and cheers


I was looking for something similar (XML to DTD inference) but I didn't
find anything related in python. Trang
(http://www.thaiopensource.com/relaxng/trang-manual.html#introduction),
on the other hand seems impressive after a few non-trivial tests. It
would be neat to have it ported in python, at least the inference part.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: XML Tree Discovery (script, tool, __?)

[George Sakkis]

<[EMAIL PROTECTED]> wrote:

> """
> I was looking for something similar (XML to DTD inference) but I didn't
> find anything related in python. Trang
> (http://www.thaiopensource.com/relaxng/trang-manual.html#introduction),
> on the other hand seems impressive after a few non-trivial tests. It
> would be neat to have it ported in python, at least the inference part.
>
> """
>
> If you're OK with RELAX NG rather than DTD as the schema output
> (probably a good idea if you're using namespaces), consider
> Examplotron, which I've used on many such production tasks.
>
> http://www-128.ibm.com/developerworks/xml/library/x-xmptron/
>
> It's XSLT rather than Python, but the good news is that XSLT is easy to
> invoke from Python using tools such as 4Suite.
>
> http://uche.ogbuji.net/tech/akara/nodes/2003-01-01/python-xslt

Neat, though non-trivial XSLT makes my head spin. Just for kicks, I
rewrote in python Michael Kay's DTDGenerator
(http://saxon.sourceforge.net/dtdgen.html), though as the original it
has several limitations on the accuracy of the inferred DTD.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Recursive generators and backtracking search

[George Sakkis]

"Talin" <[EMAIL PROTECTED]> wrote:

> I've been using generators to implement backtracking search for a while
> now. Unfortunately, my code is large and complex enough (doing
> unification on math expressions) that its hard to post a simple
> example. So I decided to look for a simpler problem that could be used
> to demonstrate the technique that I am talking about.

Here are two even simpler problems that are solved elegantly (though
not necessarily efficiently) with recursive generators:

def cartesian_product(*sequences):
'''Iterate over the elements of the cartesian product of zero or
more sequences.

>>> for x in cartesian_product(range(3), 'a', (3.25,-1.2)):
... print x
(0, 'a', 3.25)
(0, 'a', -1.2)
(1, 'a', 3.25)
(1, 'a', -1.2)
(2, 'a', 3.25)
(2, 'a', -1.2)
'''
if not sequences:
yield ()
else:
for item in sequences[0]:
head = (item,)
for tail in cartesian_product(*sequences[1:]):
yield head + tail


def powerset(iterable):
'''Iterate over all subsets of an iterable.

>>> for s in powerset('abc'):
... print s
frozenset([])
frozenset(['a'])
frozenset(['b'])
frozenset(['a', 'b'])
frozenset(['c'])
frozenset(['a', 'c'])
frozenset(['c', 'b'])
frozenset(['a', 'c', 'b'])
'''
yield frozenset()
for s in _powerset(iter(iterable)):
yield s

def _powerset(iterator):
first = frozenset([iterator.next()])
yield first
for s in _powerset(iterator):
yield s
yield s | first


George

-- 
http://mail.python.org/mailman/listinfo/python-list

Attributes of builtin/extension objects

[George Sakkis]

Hi all,

I have a few questions on object introspection of builtins and
extension modules. Here's an example:

>>> from datetime import date
>>> d=date(2003,1,23)
>>> d.__dict__
Traceback (most recent call last):
  File "", line 1, in ?
AttributeError: 'datetime.date' object has no attribute '__dict__'
>>> d.__slots__
Traceback (most recent call last):
  File "", line 1, in ?
AttributeError: 'datetime.date' object has no attribute '__slots__'
>>> dir(d)
['__add__', '__class__', '__delattr__', '__doc__', '__eq__', '__ge__',
'__getattribute__', '__gt__', '__hash__', '__init__', '__le__',
'__lt__', '__ne__', '__new__', '__radd__', '__reduce__',
'__reduce_ex__', '__repr__', '__rsub__', '__setattr__', '__str__',
'__sub__', 'ctime', 'day', 'fromordinal', 'fromtimestamp',
'isocalendar', 'isoformat', 'isoweekday', 'max', 'min', 'month',
'replace', 'resolution', 'strftime', 'timetuple', 'today', 'toordinal',
'weekday', 'year']

- Where do the attributes of a datetime.date instance live if it has
neither a __dict__ nor __slots__ ?
- How does dir() determine them ?
- dir() returns the attributes of the instance itself, its class and
its ancestor classes. Is there a way to determine the attributes of the
instance alone ?

TIA,
George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Perl XML::Simple and Data::Dumper - exists in Python?

[George Sakkis]

"Fredrik Lundh" <[EMAIL PROTECTED]> wrote:

> Miguel Manso wrote:
>
> > Can you point me out to Python solutions for:
> >
> > 1) Perl's Data::Dumper
> >
> > It dumps any perl variable to the stdout in a "readable" way.
>
> >>> import pprint
> >>> help(pprint)
>
> > 2) Perl's XML::Simple
> >
> > It maps a XML file into a Perl data structure.
>
> some alternatives:
>
> http://www.effbot.org/tags/elementtree
> http://article.gmane.org/gmane.comp.python.tutor/24986
> http://www.crummy.com/software/BeautifulSoup/
> http://www.aaronsw.com/2002/xmltramp/
> http://www.xml.com/pub/a/2005/01/19/amara.html

One more: http://www.xml.com/pub/a/2003/07/02/py-xml.html

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A Tcl/Tk programmer learns Python--any advice?

[George Sakkis]

"Kevin Walzer" <[EMAIL PROTECTED]> wrote:

> I've gotten all the approropriate resources for learning Python (docs,
> books, tutorials), so my question is this: are there any "gotchas" that
> Tcl programmers often encounter in learning Python? I'm thinking
> specifically about habits that may require "unlearning," for instance,
> such as grokking object orientation (Tcl procedures are now embedded
> deep in my brain).

I don't know Tcl, but python doesn't force you to be OO; you can write
100% procedural code if you want to. OTOH, you'll probably need other
people's code that is OO, so at the very least you'll have to be able
to read and use it. Fortunately, using an existing OO module/library is
much easier than designing and writing it, so you can get away with it
with little effort.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Map of email origins to Python list

[George Sakkis]

"Jorge Godoy" <[EMAIL PROTECTED]>:

> H...  I don't see mine listed there: I'm in South America, Brasil.  More
> specifically in Curitiba, Paraná, Brasil. :-)

That's funny; I was looking for mine and I stumbled across yours at
Piscataway, NJ, US. :-)

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: which feature of python do you like most?

[George Sakkis]

<[EMAIL PROTECTED]> wrote:

> [snipped]
>
> i've read a few chapters of a python tutorial book. The things are much like 
> the perl one.
>
> I have no idea why people are so facinating with python.

Because they can write a program or module longer than 100 lines and be
actually able to read, debug, refactor or extend it after 6 months.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [ x for x in xrange(10) when p(x) ]

[George Sakkis]

"Alex Martelli" wrote:

> [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
> > FP functions like dropwhile/takewhile etc.
>
> No way -- the itertools module is and remains a PRECIOUS resource.
> If you want an iterator rather than a list, itertools.ifilter is quite
> appropriate here.

What about the future of itertools in python 3K ? IIRC, several
functions and methods that currently return lists are going to return
iterators. Could this imply that itertools.(imap/ifilter/izip) will
take the place of map/filter/zip as builtins ?

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [ x for x in xrange(10) when p(x) ]

[George Sakkis]

"[EMAIL PROTECTED]" wrote:

> Alex Martelli wrote:
> > This becomes a valid list comprehension by writing 'if' instead of
> > 'when'.
>
> valid, yes. efficient, I am not sure.
>
> [ x for x in xrange(1000) if p(x) ]
>
> means I need to go through the whole range even if p = lambda x: x < 2

Itertools is your friend in this case:
>>> from itertools import takewhile
>>> list(takewhile(p, xrange(1000)))
[0, 1]

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [ x for x in xrange(10) when p(x) ]

[George Sakkis]

"Steve Holden" <[EMAIL PROTECTED]> wrote:

> George Sakkis wrote:
> > Itertools is your friend in this case:
> >
> >>>>from itertools import takewhile
> >>>>list(takewhile(p, xrange(1000)))
> >
> > [0, 1]
>
> Maybe, but the code also implies an esoteric knowledge that the trught
> value of the predicate is monotonically decreasing (in a Boolean sense).
> This would not be true if (e.g.) p = lambda x: x % 2 == 0. So while
> itertools.takewhile can save you unnecessary computations, such savings
> rely on provable conditions of the predicate which are frequently false.

Right, it wasn't intended as a general solution for any p. For p =
lambda x: x % 2 == 0, you do have to iterate till the iterator is
consumed anyway, so a list comprehension is not less efficient (unless
you replaced it with xrange(0,1000,2), which again implies an
esoteric knowledge of p; as usually, there's no free meal).

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What do you use as symbols for Python ?

[George Sakkis]

"Erik Max Francis" <[EMAIL PROTECTED]> wrote:

> Pierre Barbier de Reuille wrote:
>
> > When you need some symbols in your program, what do you use in Python ?
> >
> > For example, an object get a state. This state is more readable if
> > expressed as a symbols, for example "opened", "closed", "error".
> > Typically, in C or C++, I would use an enum for that:
> > enum OBJECT_STATE
> > {
> >   opened, closed, error
> > }
>
> OPENED, CLOSED, ERROR = range(3)
>
> object.state = OPENED

Or if you want something closer to real enumerations, there are several
recipes in the cookbook.
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413486 seems to
be pretty good according to the ratings.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: XML Tree Discovery (script, tool, __?)

[George Sakkis]

I somehow missed this reply posted two weeks ago.

<[EMAIL PROTECTED]> wrote:

> "Neat, though non-trivial XSLT makes my head spin."
>
> Well, you don't have to know XSLT at all to use the Examplotron
> transform, although I can understand wanting to understand and hack
> what you're using.
>
> "Just for kicks, I
> rewrote in python Michael Kay's DTDGenerator
> (http://saxon.sourceforge.net/dtdgen.html), though as the original it
> has several limitations on the accuracy of the inferred DTD. "
>
> Ah.  Cool.  Got a link?

Not a permanent one, but I posted it at
http://rafb.net/paste/results/rbxGWw37.html if you want to grab it.
Season to taste.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Inheritance in nested classes

[George Sakkis]

"Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:

> But I hope you are aware that nested classes aren't quite the same as
> they are in java, are you?

Actually they are more like java's static inner classes. If you want to
simulate non-static inner classes in python, you may check the recipe
here:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/409366.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Immutable instances, constant values

[George Sakkis]

Steven D'Aprano wrote:

> Yes, that would be how I interpret constants: You want a name which can't
> be re-bound to something else.
>
> One work-around is to use the convention of writing the name in all caps:
>
> CONSTANT = some_value
>
> and then trust that your module user doesn't rebind CONSTANT. I'd like to
> see support for something a little stronger than just "cross your fingers
> and hope", without necessarily going all the way to Pascal's full
> enforcement of constants. "We're all adults here" -- if somebody *really*
> wants to rebind CONSTANT, I'm not going to stop them, but I want them to
> jump through a hoop to do it, as a reminder that the module creator thinks
> they shouldn't be doing it.

Another workaround if you tradeoff strictness with convenience is
define a CONST  metaclass in which contants are accessed as attributes
and which raises an Exception in __setattr__:

class CONST(type):
def  __new__(cls, name, bases, dict):
def __setattr__(self,attr,val):
raise AttributeError('Cannot reassign constant %s.%s'
 % (name, attr))
cls.__setattr__ = __setattr__
return type.__new__(cls, name, bases, dict)


class PhysicsConstants(object):
__metaclass__ = CONST
c = 2.9979246e+08


>>> PhysicsConstants.c = 0
AttributeError: Cannot reassign constant PhysicsConstants.c


George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: about sort and dictionary

[George Sakkis]

"Shi Mu" wrote:

> Got confused by the following code:
> >>> a
[6, 3, 1]
> >>> b
[4, 3, 1]
> >>> c

> {1: [[6, 3, 1], [4, 3, 1]], 2: [[6, 3, 1]]}
> >>> c[2].append(b.sort())
> >>> c

> {1: [[6, 3, 1], [1, 3, 4]], 2: [[6, 3, 1], None]}
> #why c can not append the sorted b??


In python 2.4, you can use the sorted() builtin instead:

c[2].append(sorted(b))

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: about sort and dictionary

[George Sakkis]

"Shi Mu" wrote:

> Got confused by the following code:
> >>> a
[6, 3, 1]
> >>> b
[4, 3, 1]
> >>> c

> {1: [[6, 3, 1], [4, 3, 1]], 2: [[6, 3, 1]]}
> >>> c[2].append(b.sort())
> >>> c

> {1: [[6, 3, 1], [1, 3, 4]], 2: [[6, 3, 1], None]}
> #why c can not append the sorted b??


In python 2.4, you can use the sorted() builtin instead:

c[2].append(sorted(b))

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Converting a flat list to a list of tuples

[George Sakkis]

"Laurent Rahuel" wrote:

> Hi,
>
> newList = zip(aList[::2], aList[1::2])
> newList
> [('a', 1), ('b', 2), ('c', 3)]
>
> Regards,
>
> Laurent

Or if aList can get very large and/or the conversion has to be
performed many times:

from itertools import islice
newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How about "pure virtual methods"?

[George Sakkis]

"Noam Raphael" <[EMAIL PROTECTED]> wrote:

> Thanks for your suggestion, but it has several problems which the added
> class solves:
>
> * This is a very long code just to write "you must implement this
> method". Having a standard way to say that is better.
> * You can instantiate the base class, which doesn't make sense.
> * You must use testing to check whether a concrete class which you
> derived from the base class really implemented all the abstract methods.
> Testing is a good thing, but it seems to me that when the code specifies
> exactly what should happen, and it doesn't make sense for it not to
> happen, there's no point in having a separate test for it.


Here's a more refined implementation of the one posted before that
addresses these issues. It defines 'abstractclass', which would be
a nice class decoraror when (and if) class decorators make it into
the language. The extra benefit compared to the MustImplement metaclass
is that it allows a class with no abstract methods to be defined as abstract.
This makes no sense if one defines an abstract class as "a class with one
or more abstract (deferred) methods". Perhaps a more useful and general
definition of an abstract class is "a class that is not allowed to be 
instantiated".
Deferred methods are one reason for making a class uninstantiable, but it's
not the only one. Sometimes it is useful to define a base class that provides
a default implementation of a full protocol, yet disallow its direct 
instantiation,
as in the example below.

George

#===
# test_abstract.py

import unittest
from abstract import abstractclass

class AbstractTestCase(unittest.TestCase):

def test_no_abstractmethods(self):
class SomeBase(object):
# This class has no abstract methods; yet calling foo() or bar()
# on an instance of this class would cause infinite recursion.
# Hence it is defined as abstract, and its concrete subclasses
# should override at least one of (foo,bar) in a way that breaks
# the recursion.
def __init__(self, x):
self._x = x
def foo(self, y):
return self.bar(self._x + y)
def bar(self, y):
return self.foo(self._x - y)
SomeBase = abstractclass(SomeBase)

class Derived(SomeBase):
def __init__(self,x):
SomeBase.__init__(self,x)
def foo(self,y):
return self._x * y

self.assertRaises(NotImplementedError, SomeBase, 5)
self.assertEquals(Derived(5).bar(2), 15)

if __name__ == '__main__':
unittest.main()

#===
# abstract.py

import inspect

__all__ = ["abstractclass", "abstractmethod", "AbstractCheckMeta"]


def abstractclass(cls):
'''Make a class abstract.

Example::
# hopefully class decorators will be supported in python 2.x
# for some x, x>4
[EMAIL PROTECTED]
class SomeBase(object):
@abstractmethod
def function(self):
"""Implement me"""
# the only way as of python 2.4
SomeBase = abstractclass(SomeBase)

@param cls: A new-style class object.
@return: A surrogate of C{cls} that behaves as abstract. The returned
class raises NotImplementedError if attempted to be instantiated
directly; still its subclasses may call its __init__. A subclass of
the returned class is also abstract if it has one or more abstract
methods, or if it is also explicitly decorated by this function. A
method is declared abstract by being assigned to NotImplemented (or
decorated by L{abstractmethod}).
@raise TypeError: If there is a metaclass conflict between C{type(cls)}
and L{AbstractCheckMeta}, or if C{cls} has an C{__abstractmethods__}
attribute.
'''
# check if cls has AbstractCheckMeta (or a subtype) for metaclass
metaclass = type(cls)
if not issubclass(metaclass, AbstractCheckMeta):
# it doesn't; try to make AbstractCheckMeta its metaclass by
# inheriting from _AbstractCheck
cls = metaclass(cls.__name__, (_AbstractCheck,) + cls.__bases__,
dict(cls.__dict__))
# replace __init__ with a proxy ensuring that __init__ is called by a
# subclass (but not directly)
old_init = getattr(cls,'__init__',None)
def new_init(self,*args,**kwds):
if self.__class__ is cls:
raise NotImplementedError("%s is an abstract class" % cls.__name__)
if old_init is not None:
old_init(self,*args,**kwds)
setattr(cls,'__init__',new_init)
return cls


def abstractmethod(function):
'''A method decorator for those who prefer the parameters declared.'''
return NotImplemented


class AbstractCheckMeta(type):
'''A metaclass to detect instantiation of abstra

Re: Pre/Postconditions with decorators

[George Sakkis]

"Robert Brewer" <[EMAIL PROTECTED]> wrote:

> Ian Bicking was just talking about @require decorators:
> http://blog.ianbicking.org/already-under-our-noses.html
>
> @require(int, int)
> def gcd(a, b):
> ...
>
> If we made a "checker" module for such things in the stdlib, we could
> write most of that:
>
> from checker import *
>
> @args((list, itemsnotNone, ))
> @returns((any, ))
> def my_sum(seq):
> tmp=0
> for element in seq:
> tmp+=element
> return tmp


I recently wrote a similar checker module; apart from regular python types and 
classes, it supports
'parameterized types', such as 'list of ints', 'container of floats', 'mapping 
with str keys and
tuples of int values', etc. Here's a demo:

# >>> from typecheck import *
# >>> @returnType(listOf(int, size=3))
# >>> @argTypes(x=str, y=containerOf(int))
# ... def f(x,y):
# ... return [len(x)] + list(y)
# >>> f('1',[2,3])
# [1, 2, 3]
# >>> f('1',(2,3))
# [1, 2, 3]
# >>> f(1,[2,3])
# (...)
# TypeError: str expected (int given)
# >>> f('1',[2,'3'])
# (...)
# TypeError: container expected ([2, '3'] given)
# >>> f('1',[2,3,4])
# (...)
# TypeError: Container of size 3 expected ([1, 2, 3, 4] given)

I can make it available somewhere (e.g. Vaults of Parnassus) if there's enough 
interest.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre/Postconditions with decorators

[George Sakkis]

> Hi George,
> it would be nice to see how you have tackled
> the task.
> Maybe we will have a checker
> module in Python one day... ;-)

I posted my module on http://rafb.net/paste/results/voZYTG78.html and its unit 
test on
http://rafb.net/paste/results/MYxMQW95.html. Any feedback will be appreciated.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre/Postconditions with decorators

[George Sakkis]

"Stephen Thorne" <[EMAIL PROTECTED]> wrote:

> Unresolved Problems:
> 1) How do you handle duck types, i.e. a method that accepts StringIO,
> cStringIO or any other object that has a .readlines(), .seek() and
> .read() method?

I think the 'proper' way of checking duck types (among other things) is by 
defining appropriate
protocols; check pyProtocols (http://peak.telecommunity.com/PyProtocols.html) 
for details.

> 2) How do you turn off the type checking for production code?

Currently I don't, but that's easy to do if desired by checking __debug__ in 
the decorator and
return the passed function if it is false:

def params(**checkedArgs):
if not __debug__:
return lambda function: function
# else return a typechecked proxy of function
(...)


> Otherwise I quite like it. Please publish it somewhere.

Thanks. I'll post an entry on Vaults of Parnassus as soon as I publish it.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Tuple slices

[George Sakkis]

Why does slicing a tuple returns a new tuple instead of a view of the existing 
one, given that
tuples are immutable ? I ended up writing a custom ImmutableSequence class that 
does this, but I
wonder why it is not implemented for tuples.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tuple slices

[George Sakkis]

"Fredrik Lundh" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
> Steven Bethard wrote:
>
> >a = 1, 2, 3
> >b = a[:]
> >a is b
> >> True
> >
> > My impression was that full tuple copies didn't actually copy, but that 
> > slicing a subset of a
> > tuple might.  Not exactly sure how to test this, but:
> >
> > py> a = 1, 2, 3
> > py> a[:2] is a[:2]
> > False
>
> yup.  and to figure out why things are done this way, consider this case:
>
> >>> a = give_me_a_huge_tuple()
> >>> len(a)
> (a rather large number)
> >>> b = a[:2]
> >>> del a
>
> (IIRC, I proposed to add "substrings" when I implemented the Unicode string
> type, but that idea was rejected, for the very same "and how do you get rid of
> the original object" reason)
>
> 

Fair enough. So perhaps the question is whether such cases are more regular 
than something like:
a = give_me_a_huge_tuple()
slices = [a[i:j] for i in xrange(len(a)) for j in xrange(i+1, len(a)+1)]

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tuple slices

[George Sakkis]

"Peter Hansen" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> George Sakkis wrote:
> > Fair enough. So perhaps the question is whether such cases are more regular 
> > than something like:
> > a = give_me_a_huge_tuple()
> > slices = [a[i:j] for i in xrange(len(a)) for j in xrange(i+1, len(a)+1)]
>
> I believe the general usage of tuples tends to mean that
> "give_me_a_huge_tuple()" just doesn't happen except with
> those who insist on using tuples as though they were nothing
> other than read-only lists.
>
> If you use a tuple the way it was apparently intended, you
> are extraordinarily unlikely to find yourself with a
> huge one requiring slicing in such a way that you care
> whether it is a "view" or a new object.
>
> -Peter

Actually my initial motivation was not a huge tuple I had to slice many times. 
It was something much
less extraordinarily unlikely, a recursive function with a sequence parameter:

def foo(sequence):
# base_case
# do_stuff()
combine(foo(sequence[:n]),
  foo(sequence[n:]))

Having each slice be a view of the original sequence instead of a fresh copy 
would be a Good Thing
(tm).

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tuple slices

[George Sakkis]

"Terry Reedy" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
>
> Aside from the problem of not being able to delete the underlying object,
> the view object for a tuple would have to be a new type of object with a
> new set of methods.

It *could*, but it doesn't have to. One can represent a view as essentially an 
object with a pointer
to a memory buffer and a (start,stop,step) triple. Then a "real tuple" is just 
a "view" with the
triple being (0, len(sequence), 1).

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tuple slices

[George Sakkis]

"Jeff Shannon" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> [My newsreader crapped out on sending this; apologies if it appears
> twice.]
>
> George Sakkis wrote:
>
> > "Terry Reedy" <[EMAIL PROTECTED]> wrote in message
> > news:[EMAIL PROTECTED]
> >
> >>Aside from the problem of not being able to delete the underlying object,
> >>the view object for a tuple would have to be a new type of object with a
> >>new set of methods.
> >
> > It *could*, but it doesn't have to. One can represent a view as essentially 
> > an object with a
pointer
> > to a memory buffer and a (start,stop,step) triple. Then a "real tuple" is 
> > just a "view" with the
> > triple being (0, len(sequence), 1).
>
> Except that that's not how Python tuples *are* constructed, and it'd
> be a pretty big deal to redo the architecture of all tuples to support
> this relatively special case.
>
> Even if this re-architecting were done, you're still constructing a
> new object -- the difference is that you're creating this
> (start,stop,step) triple instead of duplicating a set of PyObject*
> pointers, and then doing math based on those values instead of
> straightforward pointer access.  I'm not at all convinced that this
> really saves you a significant amount for tuple slices (really, you're
> still constructing a new tuple anyhow, aren't you?), and it's going to
> cost a bit in both execution time and complexity in the common case
> (accessing tuples without slicing).  If there's a big cost in object
> construction, it's probably going to be the memory allocation, and for
> a reasonable tuple the size of the memory required is not going to
> significantly affect the allocation time.
>
> Jeff Shannon
> Technician/Programmer
> Credit International
>

You're probably right about the allocation time, but my main concern is the 
memory required for each
slice, which can be O(n) wrt the length of the whole tuple. I would like this 
to be constant (at
least if there was a way around to the problem of deleting the underlying 
sequence).

George



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tuple slices

[George Sakkis]

An iterator is perfectly ok if all you want is to iterate over the
elements of a view, but as you noted, iterators are less flexible than
the underlying sequence. The view should be (or at least appear)
identical in functionality (i.e. public methods) with its underlying
sequence.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tuple slices

[George Sakkis]

"Terry Reedy" <[EMAIL PROTECTED]> wrote in message 
news:mailman.1308.1106688018.22381.python-
>
> Unless you are GvR or Tim Peters,

Actually I am the OP. I posted my previous mail from Google groups, which for 
some reason put my
email instead of my name; it should be ok now.

> throwing 'pythonic' at me doesn't cut it
> with me, especially when you use it so shallowly.  The current Pythonic
> meaning of 'slice', as defined by GvR and implemented in core Python
> sequence objects and documented in the reference manuals and reiterated by
> GvR on PyDev in just the last day, is to make an independent in-memory
> #copy# of the indicated part of a sequence.  Both aspects are intentional
> and desired features, not accidents.

Thanks for the info; a citation that supports your claim that the in-memory 
copy is part of the
*specification* of a tuple slice -- and not a choice that is subject to the 
implementation -- would
be useful. Note that I'm talking only about slices of *tuples* (or any 
immutable sequence for that
matter) here, not all slices. As for the "pythonic", I mentioned it as a 
loosely speaking synonym to
"simpler" or "more intuitive"; I apologize if this term has religious 
connotations in cl.py.

> Yes, slicing, even in the Pythonic sense, may well simplify the OP's
> algorithm (of which he gave almost no detail), but the whole point of this
> thread is that he does not want to do that (make copy slices).  While he
> might shortsightedly think that he wants Guido to redefine slice and
> replace the current implementation of tuple with a more spacious, possibly
> slower one that would allow that definition, that will not solve his
> current problem, if indeed he has one.

I fail to understand where does your strongly negative tone come from; 
certainly not from my posts.
I asked a simple question and I was expecting a simple answer, not defending 
myself from a
hypothetical shortsighted suggestion to Guido. Thankfully I got one (and only 
so far) good reason
for the current implementation from Fredrik Lundh, namely the reference to the 
original object.

> As George Sakkis the OP noted, the essential data constituting a contiguous
> section view are the underlying sequence and two position markers.  Whether
> one works with these directly or packages them into a tuple or user class
> instance is a matter of relative conveniences.  As it turns out, I was

Honestly, I can't imagine a case where supplying these three associated data 
packaged is *less*
convenient than spelling them out explicitly.

> thinking about the design choices involved in a generic sequence view class
> just the morning before reading the original post.  But I have no idea
> whether GS's foo function would justify the added overhead of such a thing.

This is not the point; that function was just the motivation for questioning 
the current tuple slice
implementation. I wouldn't start this thread in the first place if I didn't 
have the impression that
tuple views would be beneficial for many (most?) cases.

> It partly depends on what he wishes to optimize, which I asked about, but
> have not yet seen an answer about.

Are you sure you read the whole thread ? I replied explicitly on this to Jeff 
Shannon:

"You're probably right about the allocation time, but my main concern is the 
memory required for
each slice, which can be O(n) wrt the length of the whole tuple. I would like 
this to be constant
(at least if there was a way around to the problem of deleting the underlying 
sequence)."

> So I suggested the simplest approach that would work.  And that, to me, *is* 
> pythonic!

Simplest to whom ? The user or the py-dev guy that implements tuples ? It 
sounds as if you have the
second in mind.

> Terry J. Reedy
>

George



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tuple slices

[George Sakkis]

"Jeff Shannon" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> George Sakkis wrote:
>
> > An iterator is perfectly ok if all you want is to iterate over the
> > elements of a view, but as you noted, iterators are less flexible than
> > the underlying sequence. The view should be (or at least appear)
> > identical in functionality (i.e. public methods) with its underlying
> > sequence.
>
> So, what problem is it, exactly, that you think you'd solve by making
> tuple slices a view rather than a copy?
>
> As I see it, you get the *possibility* of saving a few bytes (which

It all comes down on what you mean by "a few bytes". Since many (most?) slices 
are linear wrt to the
original sequence's length, it is not hard to think of algorithms that involve 
the creation of
*many* slices (e.g. most recursive divide-and-conquer algorithms). Implementing 
these using slices
simply does not scale as the input sequence gets larger. Of course, you can 
always use the standard
C/C++ approach and pass the original sequence along with the (start,stop,step) 
indices of the slice,
as Terry Reedy mentioned, but then you lose in expressiveness.

> may go in the other direction) at a cost of complexity and speed.  You
> have greater dependence of internal objects on each other, you can't
> get rid of the original tuple while any slice views of it exist, you
> gain nothing in the way of syntax/usage simplicity...  so what's the
> point?
>
> To my mind, one of the best things about Python is that (for the most
> part) I don't have to worry about actual memory layout of objects.  I
> don't *care* how tuples are implemented, they just work.  It seems to
> me that you're saying that they work perfectly fine as-is, but that
> you have a problem with the implementation that the language tries its
> best to let you not care about.  Is this purely abstract philosophy?

No, it's called "scalability", and it's not purely abstract philosophy AFAIK. I 
fully agree that for
the most part you don't have to care about it, and I'm grateful that python 
does all its magic
trasparently. However if you do care about it, and at the same time you are 
unwilling to sacrifice
the elegance of slices, the current implementation is not ideal.

> Jeff Shannon
> Technician/Programmer
> Credit International

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tuple slices

[George Sakkis]

"Bengt Richter" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> On Wed, 26 Jan 2005 11:55:59 -0800, jfj <[EMAIL PROTECTED]> wrote:
>
> >Jeff Shannon wrote:
> >
> >>
> >>
> >> So, what problem is it, exactly, that you think you'd solve by making
> >> tuple slices a view rather than a copy?
> >>
> >
> >I think views are good for
> >  1) saving memory
> >  2) saving time (as you don't have to copy the elements into the new tuple)
> >
> >And they are worth it. However, (as in other cases with slicing), it is
> >very easy and fast to create a view for a slice with the default step
> >'1', while it's a PITA and totally not worth it to create a view for a
> >slice with non default step. I think it would be good to:
> >
> >  if slice_step == 1
> >create_view
> >  else
> >create_new_tuple
> >
> >Actually, i think that slices with step, is a bad feature in general
> >and i think I will write a PEP to suggest their removal in python3k.
> >
> What's the big deal with other than 1 steps? It is just adjusting a few 
> numbers
> so that you can either index the new virtual slice with an integer and return 
> the
> element, in which case the index into the original tuple will be
> someoffset+i*somefactor once you get past the limit checks for the virtual
> slice. By the same token, transforming a few numbers of one virtual slice
> into similar numbers for a a new virtual slice of that shouldn't be rocket 
> science.
> And it wouldn't have to be done more than once.  Don't have time to do it now,
> but there are plenty around here that could, I'm sure.
>
> Regards,
> Bengt Richter

Here's my (undocumented) version of it: 
http://rafb.net/paste/results/HkxmHp37.html
and its unit test: http://rafb.net/paste/results/2LIInT68.html

And some useless timing comparisons (I know it's a stupid example, don't flame 
me for this):

$ python /usr/lib/python2.3/timeit.py \
-s "x=tuple(xrange(1))" \
"[x[1:-1] for n in xrange(100)]"
10 loops, best of 3: 3.84e+04 usec per loop

$ python /usr/lib/python2.3/timeit.py \
-s "from immutableseq import ImmutableSequence" \
-s "x=ImmutableSequence(xrange(1))" \
"[x[1:-1] for n in xrange(100)]"
100 loops, best of 3: 5.85e+03 usec per loop

Feel free to comment or suggest improvements.

George



-- 
http://mail.python.org/mailman/listinfo/python-list

Swig-friendly distutils.command.build_ext

[George Sakkis]

I'm using a custom extension of the build_ext distutils 'command' that 
integrates better with SWIG.
Specifically, it:

1. Adds '-I' (include) swig option for each directory containing a source file 
in the given
Extension.
2. Determines whether to add the '-c++" option by checking the source file 
extensions (even if the
'swig_cpp' and 'swig_opts' do not explicitly specify that it is a c++ wrapper).
3. Builds the swig-generated high-level python module (currently only the low 
level .dll/.so is
built).

If these changes are generally useful, I think they (c|sh)ould make it into the 
next release of
distutils.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is there no instancemethod builtin?

[George Sakkis]

"Steven Bethard" wrote:

> John Reese wrote:
> > I now do:
> >
> >   if isinstance(x, list):
> >
> > It is my understanding that this is what people do nowadays.
>
> I wouldn't go that far.  I don't have an isinstance check for lists
> anywhere in my entire codebase.  Why do you think you need to check to
> see if something is of type list?  Why don't you just use it as needed,
> and find out, e.g.:
>
> try:
>  itr = iter(x)
> except TypeError:
>  # do whatever you need to do if it's not iterable
> else:
>  # do whatever you need to do if it *is* iterable

A class of cases I've found necessary to do explicit type checking is
when a different action is taken for different types, but where duck
typing can't distinguish between them. For example, a function that
returns a string representation of an S-expression, represented as a
list of nested lists:

def s_expr(obj):
if isinstance(obj, list):
return "(%s)" % ' '.join(map(s_expr,obj))
else:
return str(obj)

>>> s_expr([1, [2,3], [4,5], "foo"])
>>> '(1 (2 3) (4 5) foo)'

Here duck typing doesn't help because the function should take the if
branch only for lists and not other iterables (e.g. strings).

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there something similar to ?: operator (C/C++) in Python?

[George Sakkis]

"Bo Peng" wrote:

> Roy Smith wrote:
>
> > Can you give us some idea of what it is that you're trying to do?  It 
> > pretty unusual to see
> > a requirement like that.
>
> def func(type_of_obj1, type_of_obj2, .):
>callfunc( [
>  type_of_obj1 and obj1a() or obj1b(),
>  type_of_obj2 and obj2a() or obj2b(),
>  
>  ])
>
> callfunc can take arbitrary number of objects whose types are determined
> by type_of_obj1 etc.

Unless the obj_i_a and obj_i_b are totally unrelated (very unlikely), I
think this might/should be refactored so that the decision between each
obj_i_a and obj_i_b is done in a separate obj_i_factory callable. If
you provide an example or two of specific triples (type_of_obj_i,
obj_i_a, obj_i_b), we'll have a better idea of whether factories are an
overkill in this case or not.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is there no instancemethod builtin?

[George Sakkis]

"Michele Simionato" wrote:

> In this case I have used hasattr(obj, "__iter__") instead of
> isinstance(obj, list)
> (strings are iterable but do not have __iter__ method). I think hasattr
> is much better
> since it works for tuples and custom iterables too.

The fact that strings don't have __iter__ is an implementation detail
(I can't think of any reason other than historic and perhaps backwards
compatibility for this; iterables should IMHO by definition be exactly
the objects with __iter__). Also, what if tuples and any other
iterables except for lists are considered atomic ? An implementation of
s-expressions would use a single data structure for the nesting (e.g. a
builtin list or a custom linked list) and objects of all other types
should be considered atoms, even if they happen to be iterable.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is there no instancemethod builtin?

[George Sakkis]

Michele Simionato wrote:
> I think strings do not have __iter__ on purpose, exactly to distinguish
> them from other iterables, since sometimes it is nice to consider them
> atomic, but I am not sure of this. You should ask the developers. Anyway, the
> right definition of iterable is (as I was told) "an object X such that
> iter(X) does not throw an exception".

Hmm.. not a very insightful definition unless someone knows the
implementation of iter().

> Objects following the __getitem__ protocol - such as strings - are iterables
> even if they do not have an __iter__ method.

It would be more uniform if the default 'type' metaclass added an
__iter__ method to classes that define __getitem__ but not __iter__
with something like:

from itertools import count

def __iter__(self):
for i in count():
try: yield self[i]
except IndexError: raise StopIteration

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is there no instancemethod builtin?

[George Sakkis]

"John Roth" wrote:

> Unfortunately it doesn't work: getitem can be defined for
> a class that acts like a dictionary: that is, the items are
> not integers, let alone integers that extend in a strict
> sequence from 0.

This is true, but that's the current behaviour of iterators for classes
that define __getitem__ without __iter__:

class AnIterable(object):
def __init__(self, it): self._it = it
def __getitem__(self,i): return self._it[i]

>>> x = AnIterable(dict(a=1,b=2))
>>> list(x)

KeyError: 0


George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Couple functions I need, assuming they exist?

[George Sakkis]

> I assume you mean translating something like '100' to '1,000,000'?
> I don't know of an existing function that does this, but here's a
> relatively simple implementation:
>
> py> import itertools as it
> py> def add_commas(s):
> ... rev_chars = it.chain(s[::-1], it.repeat('', 2))
> ... return ','.join(''.join(three_digits)
> ... for three_digits
> ... in it.izip(*[rev_chars]*3))[::-1]
> ...

Or for an equivalent less cryptic (IMHO) recipe:

def num2str(num):
'''Return a string representation of a number with the thousands
being delimited.

>>> num2str(65837)
'65,837'
>>> num2str(6582942)
'6,582,942'
>>> num2str(23)
'23'
>>> num2str(-1934)
'-1,934'
'''
parts = []
div = abs(num)
while True:
div,mod = divmod(div,1000)
parts.append(mod)
if not div:
if num < 0: parts[-1] *= -1
return ','.join(str(part) for part in reversed(parts))


Regards,
George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Back to the future - python to C++ advice wanted

[George Sakkis]

"Kay Schluehr" wrote:

> I recommend studying C++ idioms carefully.
>
> http://www1.bell-labs.com/user/cope/Patterns/C++Idioms/EuroPLoP98.html

Thanks for the link; very useful indeed.

> If Georges starts on greenfields he may have a look at Qt and it's
> object library which is not only concerned with widgets.
>
> http://doc.trolltech.com/3.3/
>
> BOOST is more high brow and I guess that it compiles slow because it
> uses templates extensively. Template metaprogramming as a compile time
> language was a funny discovery. Here is some prove of it's
> capabilities:
>
> http://osl.iu.edu/~tveldhui/papers/2003/turing.pdf

Many thanks to Kay and Bruno for suggesting Boost; I browsed through
its numerous libraries and they're quite impressive ! They seem
indispensable, especially for python (or other very high level
language) programmers going back to C++. Some libraries that seem to be
very relevant to pythoneers are:

- any: brings dynamic typing in C++
- tuple; 'nuff said
- iterator: out-of-the-box equivalents of
itertools.{imap,ifilter,izip,count}, reversed(), and others not
existing or applicable in python
- tokenizer, string_algo and regex: similar functionality to str.* and
re.*
- bind, mem_fn, function, functional, lambda: first class callables,
currying, higher order (functional) programming
- assign: syntactic sugar through operator overloading for (relatively)
readable container initialization:
map next = map_list_of(1,2)(2,3)(3,4)(4,5)(5,6);
is actually valid and equivalent to
next = dict([(1,2), (2,3), (3,4), (4,5), (5,6)])
- many, many more goodies, with or without respective standard python
equivalent (threads, graphs, math utils, serialization,
metaprogramming, etc).
- and last but not least, Boost.Python. I don't think it's just a
coincidence that among all languages they chose Python to make
interoperable with C++ :-)

Thanks again,
George

-- 
http://mail.python.org/mailman/listinfo/python-list

Compiling C++ extensions with distutils on Cygwin

[George Sakkis]

I'm trying to build a C++ extension on Cygwin, but it fails because
distutils invokes gcc instead of g++. Looking into distutils internals,
it turns out that compilation is assumed to be independent of the
target language, while linking is not (CCompiler.compile() doesn't take
a target_lang argument but CCompiler.link() does). Is there a good
reason for this ?

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Compiling C++ extensions with distutils on Cygwin

[George Sakkis]

"George Sakkis" wrote:

> I'm trying to build a C++ extension on Cygwin, but it fails because
> distutils invokes gcc instead of g++. Looking into distutils internals,
> it turns out that compilation is assumed to be independent of the
> target language, while linking is not (CCompiler.compile() doesn't take
> a target_lang argument but CCompiler.link() does). Is there a good
> reason for this ?

Is this a coincidence or what ? A patch for exactly this issue was
posted just five days ago (http://python.org/sf/1222585), so it should
be fixed in 2.5.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Does a function like isset() exist in Python?

[George Sakkis]

"Patrick Fitzsimmons" wrote:

> Hi,
>
> I'm sure I should know this, but I can't find it in the manual.
>
> Is there a function in Python like the function in PHP isset()?  It
> should take a variable name and return True or False depending on
> whether the variable is initialized.
>
> Thanks for any help,
> Patrick

There are no unitialized variables in python; if you try to access an
undefined  name, a NameError exception is raised:

try:
print "foo is", foo
except NameError:
print "foo is undefined"


To undefine a defined name, use del:
>>> foo=None
>>> print foo
None
>>> del foo
>>> print foo
NameError: name 'foo' is not defined


George

-- 
http://mail.python.org/mailman/listinfo/python-list

Profiling extension modules

[George Sakkis]

Is there a (relatively) simple way to profile an extension module,
preferably without static linking ? It compiles with 'gcc -pg' but
there are a bunch of undefined references at linking (_mcount,
_monstartup, __mcleanup); googling for them didn't bring back anything
particularly useful. Any ideas ?

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Favorite non-python language trick?

[George Sakkis]

"Joseph Garvin" wrote:

> I'm curious -- what is everyone's favorite trick from a non-python
> language? And -- why isn't it in Python?

Although it's an optimization rather than language trick, I like the
inline functions/methods in C++. There has been a thread on that in the
past (http://tinyurl.com/8ljv5) and most consider it as useless and/or
very hard to implement in python without major changes in the language
(mainly because it would introduce 'compile-time' lookup of callables
instead of runtime, as it is now). Still it might be useful to have for
time-critical situations, assuming that other optimization solutions
(psyco, pyrex, weave) are not applicable.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: a dictionary from a list

[George Sakkis]

"Rocco Moretti" wrote:

> Are you sure you need a dictionary? You may want to look at the Set
> module instead, if the values aren't important.

Set is the name of the type in the module sets, introduced in 2.3.
Since 2.4 you can use the builtin set type. Here's the import snippet
that works for 2.3 or later:

try: set
except NameError:
from sets import Set as set

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How does one write a function that increments a number?

[George Sakkis]

<[EMAIL PROTECTED]> wrote:

> Wait... so this means it is impossible to write a function that
> increments an integer without turning the integer into a list?

The short answer is no you can't, because integers are immutable (as
well as floats and strings among others). The longer answer is you can
create a, say, MutableInt class whose instances behave as modifiable
integers. Most probably you don't really need this, but if you think
you do, others in the list will sketch out how.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: a dictionary from a list

[George Sakkis]

"Roy Smith" <[EMAIL PROTECTED]> wrote:

> I just re-read the documentation on the dict() constructor.  Why does it
> support keyword arguments?
>
>dict(foo="bar", baz="blah") ==> {"foo":"bar", "baz"="blah"}
>
> This smacks of creeping featurism.  Is this actually useful in real code?
> It took me several readings of the doc to understand what this was doing.
> Essentially, it's Perl's bareword syntax, and once I realized that, I
> simultaneously understood what was happening and was revolted that Python
> seems to have picked up one of Perl's most bizarre and confusing features.

The worst thing about this form of the dict constructor it's not the
syntax; I think this becomes obvious after you've seen it once. More
annoying is that it "wastes" keyword arguments that could otherwise be
used to determine the characteristics of the dict. Perhaps the most
common desired feature is setting a default value for the dict, that
would allow for instance:

wordCount = dict(default=0)
wordPos = dict(default=[])
for pos,word in enumerate(text):
wordCount[word] += 1
wordPos[word].append(pos)

Other candidate optional arguments would allow type checking (e.g.
dict(keys=int, values=list)) or performance fine-tuning (e.g.
dict(minsize = 10, maxsize = 1, average = 200)). I hope the
decision for this form of the constructor is reconsidered for python
3K.

George

-- 
http://mail.python.org/mailman/listinfo/python-list