*** ISRAELI TERRORISTS BLEW UP INDIAN TRAIN ***

[thermate2]

Eye opener research

http://www.apfn.org/apfn/WTC_STF.htm

-- 
http://mail.python.org/mailman/listinfo/python-list

RegExp performance?

[Christian Sonne]

Long story short, I'm trying to find all ISBN-10 numbers in a multiline 
string (approximately 10 pages of a normal book), and as far as I can 
tell, the *correct* thing to match would be this:
".*\D*(\d{10}|\d{9}X)\D*.*"

(it should be noted that I've removed all '-'s in the string, because 
they have a tendency to be mixed into ISBN's)

however, on my 3200+ amd64, running the following:

reISBN10 = re.compile(".*\D*(\d{10}|\d{9}X)\D*.*")
isbn10s = reISBN10.findall(contents)

(where contents is the string)

this takes about 14 minutes - and there are only one or two matches...

if I change this to match ".*[ ]*(\d{10}|\d{9}X)[ ]*.*" instead, I risk 
loosing results, but it runs in about 0.3 seconds

So what's the deal? - why would it take so long to run the correct one? 
- especially when a slight modification makes it run as fast as I'd 
expect from the beginning...


I'm sorry I cannot supply test data, in my case, it comes from 
copyrighted material - however if it proves needed, I can probably 
construct dummy data to illustrate the problem


Any and all guidance would be greatly appreciated,
kind regards
Christian Sonne

PS: be gentle - it's my first post here :-)
-- 
http://mail.python.org/mailman/listinfo/python-list

select which lib for iNet?

[Marco]

Hello everyone,

I wanta know why (Py)Qt has its own lib on socket/ sql/ openGL,etc ?

Is the lib better than standard lib?

Thank you!


-- 
LinuX Power
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: select which lib for iNet?

[Marc 'BlackJack' Rintsch]

In <[EMAIL PROTECTED]>, Marco wrote:

> I wanta know why (Py)Qt has its own lib on socket/ sql/ openGL,etc ?

That's simply because Qt has these libraries.  If you are programming in
C++ this has the advantage that the application is platform independent if
you write against the Qt APIs.  As Python also has APIs for such things
that work cross platform it might be better to use those, because you can
separate GUI and logic better.  It's hard to replace the Qt libraries with
a Gtk, wxPython or web GUI if you still need Qt for networking or database
access.

Ciao,
Marc 'BlackJack' Rintsch
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RegExp performance?

[Gabriel Genellina]

En Sun, 25 Feb 2007 05:21:49 -0300, Christian Sonne <[EMAIL PROTECTED]>  
escribió:

> Long story short, I'm trying to find all ISBN-10 numbers in a multiline
> string (approximately 10 pages of a normal book), and as far as I can
> tell, the *correct* thing to match would be this:
> ".*\D*(\d{10}|\d{9}X)\D*.*"

Why the .* at the start and end? You dont want to match those, and makes  
your regexp slow.
You didn't tell how exactly a ISBN-10 number looks like, but if you want  
to match 10 digits, or 9 digits followed by an X:
reISBN10 = re.compile("\d{10}|\d{9}X")
That is, just the () group in your expression. But perhaps this other one  
is better (I think it should be faster, but you should measure it):
reISBN10 = re.compile("\d{9}[\dX]")
("Nine digits followed by another digit or an X")

> if I change this to match ".*[ ]*(\d{10}|\d{9}X)[ ]*.*" instead, I risk
> loosing results, but it runs in about 0.3 seconds

Using my suggested expressions you might match some garbage, but not loose  
anything (except two ISBN numbers joined together without any separator in  
between). Assuming you have stripped all the "-", as you said.

> So what's the deal? - why would it take so long to run the correct one?
> - especially when a slight modification makes it run as fast as I'd
> expect from the beginning...

Those .* make the expression match a LOT of things at first, just to  
discard it in the next step.

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: select which lib for iNet?

[Gabriel Genellina]

En Sun, 25 Feb 2007 06:12:32 -0300, Marco <[EMAIL PROTECTED]> escribió:

> I wanta know why (Py)Qt has its own lib on socket/ sql/ openGL,etc ?
> Is the lib better than standard lib?

See http://www.commandprompt.com/community/pyqt/x3738
In short, because Qt already had them, and some callbacks or function  
calls may need to use them instead of the native Python classes.

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RegExp performance?

[Marc 'BlackJack' Rintsch]

In <[EMAIL PROTECTED]>, Christian Sonne wrote:

> Long story short, I'm trying to find all ISBN-10 numbers in a multiline 
> string (approximately 10 pages of a normal book), and as far as I can 
> tell, the *correct* thing to match would be this:
> ".*\D*(\d{10}|\d{9}X)\D*.*"
> 
> (it should be noted that I've removed all '-'s in the string, because 
> they have a tendency to be mixed into ISBN's)
> 
> however, on my 3200+ amd64, running the following:
> 
> reISBN10 = re.compile(".*\D*(\d{10}|\d{9}X)\D*.*")
> isbn10s = reISBN10.findall(contents)
> 
> (where contents is the string)
> 
> this takes about 14 minutes - and there are only one or two matches...

First of all try to get rid of the '.*' at both ends of the regexp.  Don't
let the re engine search for any characters that you are not interested in
anyway.

Then leave off the '*' after '\D'.  It doesn't matter if there are
multiple non-digits before or after the ISBN, there just have to be at
least one.  BTW with the star it even matches *no* non-digit too!

So the re looks like this: '\D(\d{10}|\d{9}X)\D'

Ciao,
Marc 'BlackJack' Rintsch
-- 
http://mail.python.org/mailman/listinfo/python-list

Find the first element that meets the condition

[[EMAIL PROTECTED]]

Hi,
I have a list and I want to find the first element that meets a
condition. I do not want to use 'filter', because I want to come out
of the iteration as soon as the first element is found.
I have implemented it this way, may be, there should be a built in
hiding somewhere in the standard libraries?

def exists(iterable, condition):
'''
Return the first element in iterble that meets the condition.
'''
for x in iterable:
if condition(x):
return x
raise Exception('No element meets the given condition.')



>>> exists(xrange(1000), lambda x: x>13)
14

-
Suresh

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Find the first element that meets the condition

[Troy Melhase]

> I have implemented it this way, may be, there should be a built in
> hiding somewhere in the standard libraries?

the itertools module might have what you're after.  something similar
to your example:

>>> import itertools
>>> r = iter(range(100))
>>> n = itertools.dropwhile(lambda x:x<=13, r)
>>> n.next()
14
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Find the first element that meets the condition

[Peter Otten]

[EMAIL PROTECTED] wrote:

> I have a list and I want to find the first element that meets a
> condition. I do not want to use 'filter', because I want to come out
> of the iteration as soon as the first element is found.
> I have implemented it this way, may be, there should be a built in
> hiding somewhere in the standard libraries?
> 
> def exists(iterable, condition):
> '''
> Return the first element in iterble that meets the condition.
> '''
> for x in iterable:
> if condition(x):
> return x
> raise Exception('No element meets the given condition.')
> 
> 
> 
 exists(xrange(1000), lambda x: x>13)
> 14

If you are only interested in existence you can use any() (new in Python2.5)

>>> any(x>13 for x in xrange(1000))
True

Otherwise there is itertools.ifilter() a lazy variant of the filter()
builtin:

>>> import itertools
>>> items = iter(xrange(1000)) # just to prove...
>>> itertools.ifilter(lambda x: x > 13, items).next()
14
>>> items.next() # that ifilter has consumed only the first 15 items
15

You may want to wrap the ifilter() call to get a more sensible exception,
say ValueError instead of StopIteration:

# untested
def findfirst(items, predicate=bool):
for item in itertools.ifilter(predicate, items):
return item
raise ValueError("No matching element")

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RegExp performance?

[John Machin]

On Feb 25, 7:21 pm, Christian Sonne <[EMAIL PROTECTED]> wrote:
> Long story short, I'm trying to find all ISBN-10 numbers in a multiline
> string (approximately 10 pages of a normal book), and as far as I can
> tell, the *correct* thing to match would be this:
> ".*\D*(\d{10}|\d{9}X)\D*.*"

All of those *s are making it work too hard.

Starting with your r".*\D*(\d{10}|\d{9}X)\D*.*" [you do have the
r"..." not just "", don't you?]

Step 1: Lose the .* off each end -- this is meaningless in the context
of a search() or findall() and would slow the re engine down if it
doesn't optimise it away.

r"\D*(\d{10}|\d{9}X)\D*"

Step 2: I presume that the \D* at each (remaining) end is to ensure
that you don't pick up a number with 11 or more digits. You only need
to test that your presumed ISBN is not preceded/followed by ONE
suspect character. Is ABC1234567890DEF OK? I think not; I'd use \b
instead of \D

r"\b(\d{10}|\d{9}X)\b"

Step 3: Now that we have only \b (which matches 0 characters) at each
end of the ISBN, we can lose the capturing ()

r"\b\d{10}|\d{9}X\b"

Step 4: In the case of 123456789X, it fails on the X and then scans
the 123456789 again -- a tiny waste compared to all the * stuff, but
still worth fixing.

r"\b\d{9}[0-9X]\b"

Give that a whirl and let us know how correct and how fast it is.

>
> (it should be noted that I've removed all '-'s in the string, because
> they have a tendency to be mixed into ISBN's)
>
> however, on my 3200+ amd64, running the following:
>
> reISBN10 = re.compile(".*\D*(\d{10}|\d{9}X)\D*.*")

You should really get into the habit of using raw strings with re.

> isbn10s = reISBN10.findall(contents)
>
> (where contents is the string)
>
> this takes about 14 minutes - and there are only one or two matches...

How many actual matches and how many expected matches?

Note on "and there are only one or two matches": if your data
consisted only of valid ISBNs separated by a single space or comma, it
would run much faster. It is all the quadratic/exponential mucking
about with the in-between bits that slows it down. To demonstrate
this, try timing dummy data like "1234567890 " * 100 and
"123456789X " * 100 with your various regexes, and with the step1,
step2 etc regexes above.

>
> if I change this to match ".*[ ]*(\d{10}|\d{9}X)[ ]*.*" instead, I risk
> loosing results, but it runs in about 0.3 seconds
>
> So what's the deal? - why would it take so long to run the correct one?

Because you have .*\D* in other words 0 or more occurrences of almost
anything followed by 0 or more occurrences of almost anything. Even
assuming it ignores the .*, it will find the longest possible sequence
of non-digits, then try to match the ISBN stuff. If it fails, it will
shorten that sequence of non-digits, try the ISBN stuff again, etc etc
until it matches the ISBN stuff or that sequence of non-digits is down
to zero length. It will do that for each character position in the
file contents. Why is it faster when you change \D to []? Presumably
because in your data, sequences of non-digits are longer than
sequences of spaces IOW there is less stuff to back-track over.


> - especially when a slight modification makes it run as fast as I'd
> expect from the beginning...
>
> I'm sorry I cannot supply test data, in my case, it comes from
> copyrighted material - however if it proves needed, I can probably
> construct dummy data to illustrate the problem

What you call "dummy data", I'd call "test data". You should make a
sample of "dummy data" and test that your regex is (a) finding all
ISBNs that it should (b) not reporting incorrect matches, *BEFORE*
being concerned with speed.

>
> Any and all guidance would be greatly appreciated,

For some light reading :-) borrow a copy of Friedl's book (mentioned
in the Python re docs) and read the parts on how backtracking regex
engines work.

HTH,
John

-- 
http://mail.python.org/mailman/listinfo/python-list

finding out the precision of floats

[Arnaud Delobelle]

Hi all,

I want to know the precision (number of significant digits) of a float
in a platform-independent manner.  I have scoured through the docs but
I can't find anything about it!

At the moment I use this terrible substitute:

FLOAT_PREC = repr(1.0/3).count('3')

How can I do this properly or where is relevant part of the docs?

Thanks

--
Arnaud

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there any way to automatically create a transcript of an interactive Python session?

[skip]

Jonathan> Some languages, such as Scheme, permit you to make a
Jonathan> transcript of an interactive console session. Is there a way
Jonathan> to do that in Python?

If you decide not to use IPython for some reason, you should be able to
easily whip something up based on the command history features of the
readline module or one of the interpreter classes in the code module.

Skip



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: finding out the precision of floats

[John Machin]

On Feb 25, 9:57 pm, "Arnaud Delobelle" <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I want to know the precision (number of significant digits) of a float
> in a platform-independent manner.  I have scoured through the docs but
> I can't find anything about it!
>
> At the moment I use this terrible substitute:
>
> FLOAT_PREC = repr(1.0/3).count('3')

I'm a little puzzled:

You don't seem to want a function that will tell you the actual number
of significant decimal digits in a particular number e.g.

nsig(12300.0) -> 3
nsig(0.00123400) -> 4
etc

You appear to be trying to determine what is the maximum number of
significant decimal digits afforded by the platform's implementation
of Python's float type. Is Python implemented on a platform that
*doesn't* use IEEE 754 64-bit FP as the in-memory format for floats?

Cheers,
John



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: timeout in urllib.open()

[skip]

>> I believe this can only be set globally:
>> 
>> import socket
>> socket.setdefaulttimeout(seconds)

Stefan> Uuuh this is no solution for me, because the website-checking
Stefan> tool is part of a very very big application running in an
Stefan> application server, so globally setting the timeout may break a
Stefan> lot of other things...

Can you try this patch:

http://python.org/sf/723312

?  It's against current CVS.  Also, it doesn't include urllib or urllib2
changes, just httplib.  Getting this completed would be nice.

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: finding out the precision of floats

[Arnaud Delobelle]

On Feb 25, 11:20 am, "John Machin" <[EMAIL PROTECTED]> wrote:
[...]
> I'm a little puzzled:
>
> You don't seem to want a function that will tell you the actual number
> of significant decimal digits in a particular number e.g.
>
> nsig(12300.0) -> 3
> nsig(0.00123400) -> 4
> etc
>
> You appear to be trying to determine what is the maximum number of
> significant decimal digits afforded by the platform's implementation
> of Python's float type.

Yes you are correct.

> Is Python implemented on a platform that
> *doesn't* use IEEE 754 64-bit FP as the in-memory format for floats?

I had no knowledge of IEEE 754 64-bit FP.  The python doc says that
floats are implemented using the C 'double' data type but I didn't
realise there was a standard for this accross platforms .

Thanks for clarifying this. As my question shows I am not versed in
floating point arithmetic!

Looking at the definition of IEEE 754, the mantissa is made of 53
significant binary digits, which means
53*log10(2) = 15.954589770191003 significant decimal digits
(I got 16 with my previous dodgy calculation).

Does it mean it is safe to assume that this would hold on any
platform?

--
Arnaud

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 unfit for serious work?

[skip]

Jeff> My post was entitled "Python 3.0 unfit for serious work", you just
Jeff> indicated that the Linux distros will agree with me, in order to
Jeff> be taken seriously, the distros will have to include 2.x python
Jeff> for a very long time.  If 3.0 and 2.x have any serious degree of
Jeff> incompatibility, python will be a favorite subject for religious
Jeff> rants and heated arguments for many people.  

The notion that Python 3.0 was going to fix design/implementation issues in
earlier versions of Python and thus cause some breakage has been known for a
long time, at least since Guido's UK Python talk in March 2003.  Python 2.x
released will continue to be created for some time after Python 3.0 is
released.  From PEP-3000:

I expect that there will be parallel Python 2.x and 3.x releases for
some time; the Python 2.x releases will continue for a longer time than
the traditional 2.x.y bugfix releases. Typically, we stop releasing
bugfix versions for 2.x once version 2.(x+1) has been released. But I
expect there to be at least one or two new 2.x releases even after 3.0
(final) has been released, probably well into 3.1 or 3.2. This will to
some extend depend on community demand for continued 2.x support,
acceptance and stability of 3.0, and volunteer stamina.

The whole intention is to give authors a long period of time to port to
Python 3.x.  I believe your fear is just a knee jerk reaction to the notion
that there will be some stated incompatibilities between 2.x and 3.x without
having done any investigation of the transition process.  Nobody is forcing
you to do anything right now or completely abandon your code base.  Python
2.x still has a long shelf life.  Hell, 3.0a1 isn't even out yet.  If you
hang on for a few days I'm sure Guido's keynote about Python 3 from the
PyCon just wrapping up in Dallas will be available online.  There might be
something in there of interest to you.  If you poke around a bit you will
probably find nearly live blogs from the conference as well.

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to build Hierarchies of dict's? (Prototypes in Python?)

[Toby]

Charles D Hixson wrote:
> a class whose sub-classes automatically have unique class variables of
> a determined form such that I can do a hierarchical search through them

Something like this?  
(scroll down to see the results)


# --- begin ---

class AttrSearch(object):
  @classmethod
  def getclassattrset(cls, name):
s = set()
if name in cls.__dict__:
  s.add((cls.__name__, cls.__dict__[name]))
for base in cls.__bases__:
  try:
s.update(base.getclassattrset(name))
  except AttributeError:
pass
return s

  def getattrset(self, name):
s = set()
try:
  s.add((None, self.__dict__[name]))
except KeyError:
  pass
s.update(self.__class__.getclassattrset(name))
return s

  def __getattribute__(self, name):
if name.startswith('__'): #XXX not pretty
  return object.__getattribute__(self, name)
found = AttrSearch.getattrset(self, name)
print 'Looking for "%s" in a %s instance, found %d candidates:' \
  % (name, self.__class__.__name__, len(found))
print '\n'.join([ '  %-4s %s' % x for x in found ])
print '(now choose wisely what to return)'

class A(AttrSearch):
  a = 1

class B(A):
  a = 2

class C(A):
  a = 3

class D(B, C):
  a = 4


D().a

# --- end ---


Results:

Looking for "a" in a D instance, found 4 candidates:
  A1
  B2
  C3
  D4
(now choose wisely what to return)


Toby
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CSV(???)

[skip]

David> Is there a csvlib out there somewhere?
...
David> ...Believe it or not I'm _still_ using python 1.5.7

You might see if someone has written a pure Python version of the csv module
for use in PyPy.

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: HTML Parsing

[Stefan Behnel]

John Machin wrote:
> One can even use ElementTree, if the HTML is well-formed. See below.
> However if it is as ill-formed as the sample (4th "td" element not
> closed; I've omitted it below), then the OP would be better off
> sticking with Beautiful Soup :-)

Or (as we were talking about the best of both worlds already) use lxml's HTML
parser, which is also capable of parsing pretty disgusting HTML-like tag soup.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: finding out the precision of floats

[John Machin]

On Feb 25, 11:06 pm, "Arnaud Delobelle" <[EMAIL PROTECTED]>
wrote:
> On Feb 25, 11:20 am, "John Machin" <[EMAIL PROTECTED]> wrote:
> [...]
>
> > I'm a little puzzled:
>
> > You don't seem to want a function that will tell you the actual number
> > of significant decimal digits in a particular number e.g.
>
> > nsig(12300.0) -> 3
> > nsig(0.00123400) -> 4
> > etc
>
> > You appear to be trying to determine what is the maximum number of
> > significant decimal digits afforded by the platform's implementation
> > of Python's float type.
>
> Yes you are correct.
>
> > Is Python implemented on a platform that
> > *doesn't* use IEEE 754 64-bit FP as the in-memory format for floats?
>
> I had no knowledge of IEEE 754 64-bit FP.  The python doc says that
> floats are implemented using the C 'double' data type but I didn't
> realise there was a standard for this accross platforms .
>
> Thanks for clarifying this. As my question shows I am not versed in
> floating point arithmetic!
>
> Looking at the definition of IEEE 754, the mantissa is made of 53
> significant binary digits, which means
> 53*log10(2) = 15.954589770191003 significant decimal digits
> (I got 16 with my previous dodgy calculation).
>
> Does it mean it is safe to assume that this would hold on any
> platform?
>

Evidently not; here's some documentation we both need(ed) to read:

http://docs.python.org/tut/node16.html
"""
Almost all machines today (November 2000) use IEEE-754 floating point
arithmetic, and almost all platforms map Python floats to IEEE-754
"double precision".
"""
I'm very curious to know what the exceptions were in November 2000 and
if they still exist. There is also the question of how much it matters
to you. Presuming the representation is 64 bits, even taking 3 bits
off the mantissa and donating them to the exponent leaves you with
15.05 decimal digits -- perhaps you could assume that you've got at
least 15 decimal digits.

While we're waiting for the gurus to answer, here's a routine that's
slightly less dodgy than yours:

| >>> for n in range(200):
| ... if (1.0 + 1.0/2**n) == 1.0:
| ... print n, "bits"
| ... break
| ...
| 53 bits

At least this method has no dependency on the platform's C library.
Note carefully the closing words of that tutorial section:
"""
(well, will display on any 754-conforming platform that does best-
possible input and output conversions in its C library -- yours may
not!).
"""

I hope some of this helps ...
Cheers,
John

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: finding out the precision of floats

[Arnaud Delobelle]

On Feb 25, 1:31 pm, "John Machin" <[EMAIL PROTECTED]> wrote:
> On Feb 25, 11:06 pm, "Arnaud Delobelle" <[EMAIL PROTECTED]>
> wrote:
[...]
> Evidently not; here's some documentation we both need(ed) to read:
>
> http://docs.python.org/tut/node16.html

Thanks for this link

> I'm very curious to know what the exceptions were in November 2000 and
> if they still exist. There is also the question of how much it matters
> to you. Presuming the representation is 64 bits, even taking 3 bits
> off the mantissa and donating them to the exponent leaves you with
> 15.05 decimal digits -- perhaps you could assume that you've got at
> least 15 decimal digits.

It matters to me because I prefer to avoid making assumptions in my
code!
Moreover the reason I got interested in this is because I am creating
a
Dec class (decimal numbers) and I would like that:
   * given a float x, float(Dec(x)) == x for as many values of x as
possible
   * given a decimal d, Dec(float(d)) == d for as many values of d as
possible.

(and I don't want the standard Decimal class :)

> While we're waiting for the gurus to answer, here's a routine that's
> slightly less dodgy than yours:
>
> | >>> for n in range(200):
> | ... if (1.0 + 1.0/2**n) == 1.0:
> | ... print n, "bits"
> | ... break
> | ...
> | 53 bits

Yes I have a similar routine:

|def fptest():
|"Gradually fill the mantissa of a float with 1s until running out
of bits"
|ix = 0
|fx = 0.0
|for i in range(200):
|fx = 2*fx+1
|ix = 2*ix+1
|if ix != fx:
|return i

...

>>> fptest()
53

I guess it would be OK to use this.  It's just that this 'poking into
floats'
seems a bit strange to me ;)

Thanks for your help

--
Arnaud

-- 
http://mail.python.org/mailman/listinfo/python-list

convert strings to utf-8

[Niclas]

Hi

I'm having trouble to work with the special charcters in swedish (Å Ä Ö 
å ä ö). The script is parsing and extracting information from a webpage. 
This works fine and I get all the data correctly. The information is 
then added to a rss file (using xml.dom.minidom.Document() to create the 
file), this is where it goes wrong. Letters like Å ä ö get messed up and 
the rss file does not validate. How can I convert the data to UTF-8 
without loosing the special letters?

Thanks in advance
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Endianness conversion

[Dan Sommers]

On Sat, 24 Feb 2007 17:27:12 +, Toby wrote:

> Dan Sommers wrote:
>> You could try the struct module.  If your input comes in fixed sized
>> chunks, just call struct.unpack and struct.pack once per chunk.
> 
> Thanks, but it was a bit awkward to use for big chunks.

def swapper( bytestring ):
someHs = len( bytestring ) / 2 * "H" # could check for odd-lengths?
unpackfmt = "<" + someHs
packfmt = ">" + someHs
return struct.pack( packfmt, *struct.unpack( unpackfmt, bytestring )

-- 
Dan Sommers

Atoms are not things. -- Werner Heisenberg.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: convert strings to utf-8

[Diez B. Roggisch]

Niclas schrieb:
> Hi
> 
> I'm having trouble to work with the special charcters in swedish (Å Ä Ö 
> å ä ö). The script is parsing and extracting information from a webpage. 
> This works fine and I get all the data correctly. The information is 
> then added to a rss file (using xml.dom.minidom.Document() to create the 
> file), this is where it goes wrong. Letters like Å ä ö get messed up and 
> the rss file does not validate. How can I convert the data to UTF-8 
> without loosing the special letters?

Show us code, and example text (albeit I know it is difficult to get 
that right using news/mail)

The basic idea is this:

scrapped_byte_string = scrap_the_website()

output = scrappend_byte_string.decode('website-encoding').encode('utf-8')



Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: finding out the precision of floats

[Jerry Hill]

On 25 Feb 2007 06:11:02 -0800, Arnaud Delobelle <[EMAIL PROTECTED]> wrote:
> Moreover the reason I got interested in this is because I am creating
> a Dec class (decimal numbers)

Are you familiar with Python's Decimal library?
http://docs.python.org/lib/module-decimal.html

-- 
Jerry
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Are weak refs slower than strong refs?

[shredwheat]

On Feb 24, 10:17 pm, John Nagle <[EMAIL PROTECTED]> wrote:
>Are weak refs slower than strong refs?  I've been considering making the
> "parent" links in BeautifulSoup into weak refs, so the trees will release
> immediately when they're no longer needed.  In general, all links back
> towards the root of a tree should be weak refs; this breaks the loops
> that give reference counting trouble.

I've never really benchmarked their overhead. Thinking about how they
work, I wouldn't expect a measurable difference in the time for
dereferencing the weakrefs. But internally objects must track who all
their weak reference holders are, so that will add some cost. I am
guessing if the hierarchy is built once and remains fairly static you
won't see the cost of this management. If the hierarchy is very
dynamic there will be more weakref overhead. On the other hand, you
will be freeing up the work done by the cyclic testing in the garbage
collector.

After doing similar work with weakrefs in hierarchies, I'd say it is
worth doing. If you do the work, it would be interesting to see what
the performance looks like. Be sure there is a gc.collect() in the
timing for the  original strong-ref version. :-)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: finding out the precision of floats

[Arnaud Delobelle]

On Feb 25, 3:59 pm, "Jerry Hill" <[EMAIL PROTECTED]> wrote:
> On 25 Feb 2007 06:11:02 -0800, Arnaud Delobelle <[EMAIL PROTECTED]> wrote:
>
> > Moreover the reason I got interested in this is because I am creating
> > a Dec class (decimal numbers)
>
> Are you familiar with Python's Decimal 
> library?http://docs.python.org/lib/module-decimal.html
>

Read on my post for a few more lines and you'll know that the answer
is yes :)

--
Arnaud

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: timeout in urllib.open()

[John J. Lee]

[EMAIL PROTECTED] writes:

> >> I believe this can only be set globally:
> >> 
> >> import socket
> >> socket.setdefaulttimeout(seconds)
> 
> Stefan> Uuuh this is no solution for me, because the website-checking
> Stefan> tool is part of a very very big application running in an
> Stefan> application server, so globally setting the timeout may break a
> Stefan> lot of other things...
> 
> Can you try this patch:
> 
> http://python.org/sf/723312
> 
> ?  It's against current CVS.  Also, it doesn't include urllib or urllib2
> changes, just httplib.  Getting this completed would be nice.

Great.  Added some comments, will try to write tests and urllib2 patch
later.


John
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Are weak refs slower than strong refs?

[Duncan Booth]

John Nagle <[EMAIL PROTECTED]> wrote:

> Are weak refs slower than strong refs?  I've been considering
> making the "parent" links in BeautifulSoup into weak refs, so the
> trees will release immediately when they're no longer needed.  In
> general, all links back towards the root of a tree should be weak
> refs; this breaks the loops that give reference counting trouble.

Yes, weak references are slower, but that may not be significant unless 
people are following the parent links a lot: I would expect other 
processing to make the overhead fairly insignificant.

Why not do a few timing tests with some typical use cases? Here's an 
example which does nothing much but create and follow. Typical output:

Using proxy
100 loops, best of 3: 5.6 msec per loop
Call to dummy proxy
100 loops, best of 3: 3.11 msec per loop
Without proxy call
100 loops, best of 3: 2.71 msec per loop

-- timetest.py 
from timeit import Timer, default_timer as timer
from weakref import proxy

class C:
'''Use a weak proxy (or whatever 'proxy' returns)'''
def __init__(self, parent=None):
self.parent = proxy(parent) if parent is not None else parent
self.child = None

def show_parents(self):
while self is not None:
# print "show_parents", id(self)
self = self.parent

def show_children(self):
while self is not None:
# print "show_children", id(self)
self = self.child


def t1(n):
base = C()
current = base
for i in range(n):
current.child = C(current)
current = current.child
base.show_children()
current.show_parents()

class D:
'''Strong parent reference'''
def __init__(self, parent=None):
self.parent = parent if parent is not None else parent
self.child = None

def show_parents(self):
while self is not None:
# print "show_parents", id(self)
self = self.parent

def show_children(self):
while self is not None:
# print "show_children", id(self)
self = self.child

def t2(n):
base = D()
current = base
for i in range(n):
current.child = D(current)
current = current.child
base.show_children()
current.show_parents()


def timetest(stmt):
repeat = 3
precision = 3
t = Timer(stmt, 'import __main__', timer)

# determine number so that 0.2 <= total time < 2.0
for i in range(1, 10):
number = 10**i
try:
x = t.timeit(number)
except:
t.print_exc()
return 1
if x >= 0.2:
break

try:
r = t.repeat(repeat, number)
except:
t.print_exc()
return 1
best = min(r)
print "%d loops," % number,
usec = best * 1e6 / number
if usec < 1000:
print "best of %d: %.*g usec per loop" % (repeat, precision, usec)
else:
msec = usec / 1000
if msec < 1000:
print "best of %d: %.*g msec per loop" % (repeat, precision, 
msec)
else:
sec = msec / 1000
print "best of %d: %.*g sec per loop" % (repeat, precision, 
sec)

if __name__=='__main__':
print "Using proxy"
timetest('__main__.t1(1000)')
print "Call to dummy proxy"
def proxy(x): return x
timetest('__main__.t1(1000)')
print "Without proxy call"
timetest('__main__.t2(1000)')
 -
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Referencing Items in a List of Tuples

[Paddy]

On Feb 25, 2:01 am, [EMAIL PROTECTED] wrote:
>   While working with lists of tuples is probably very common, none of my
> five Python books or a Google search tell me how to refer to specific items
> in each tuple. I find references to sorting a list of tuples, but not
> extracting tuples based on their content.
>
>   In my case, I have a list of 9 tuples. Each tuple has 30 items. The first
> two items are 3-character strings, the remaining 28 itmes are floats.
>
>   I want to create a new list from each tuple. But, I want the selection of
> tuples, and their assignment to the new list, to be based on the values of
> the first two items in each tuple.
>
>   If I try, for example, writing:
>
> for item in mainlist:
> if mainlist[item][0] == 'eco' and mainlist[item][1] == 'con':
>   ec.Append(mainlist[item][2:])
>
> python doesn't like a non-numeric index.
>
>   I would really appreciate a pointer so I can learn how to manipulate lists
> of tuples by referencing specific items in each tuple (string or float).
>
> Rich

You might also use list comprehensions to accumulate the values you
need:

ec = [ item[2:]  for item in mainlist  if item[:2] == ['eco','con'] ]

- Paddy.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Find the first element that meets the condition

[Paul Rubin]

"[EMAIL PROTECTED]" <[EMAIL PROTECTED]> writes:
> I have a list and I want to find the first element that meets a
> condition. I do not want to use 'filter', because I want to come out
> of the iteration as soon as the first element is found.
> I have implemented it this way, may be, there should be a built in
> hiding somewhere in the standard libraries?
> 
> def exists(iterable, condition):

To check for existence, use any(condition, iterable).  If you actually
want the first element, use itertools.ifilter(condition, iterable).next().
The suggestion of using dropwhile isn't so great because it actually
consumes the first match, so you have to write your condition to
stop immediately before the element you want, not always easy.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Referencing Items in a List of Tuples

[Jussi Salmela]

Paddy kirjoitti:
> On Feb 25, 2:01 am, [EMAIL PROTECTED] wrote:
>>   While working with lists of tuples is probably very common, none of my
>> five Python books or a Google search tell me how to refer to specific items
>> in each tuple. I find references to sorting a list of tuples, but not
>> extracting tuples based on their content.
>>
>>   In my case, I have a list of 9 tuples. Each tuple has 30 items. The first
>> two items are 3-character strings, the remaining 28 itmes are floats.
>>
>>   I want to create a new list from each tuple. But, I want the selection of
>> tuples, and their assignment to the new list, to be based on the values of
>> the first two items in each tuple.
>>
>>   If I try, for example, writing:
>>
>> for item in mainlist:
>> if mainlist[item][0] == 'eco' and mainlist[item][1] == 'con':
>>   ec.Append(mainlist[item][2:])
>>
>> python doesn't like a non-numeric index.
>>
>>   I would really appreciate a pointer so I can learn how to manipulate lists
>> of tuples by referencing specific items in each tuple (string or float).
>>
>> Rich
> 
> You might also use list comprehensions to accumulate the values you
> need:
> 
> ec = [ item[2:]  for item in mainlist  if item[:2] == ['eco','con'] ]
> 
> - Paddy.
> 

I'm nitpicking, but the OP has a list of tuples:

ec = [ item[2:]  for item in mainlist  if item[:2] == ('eco','con') ]


Cheers,
Jussi
-- 
http://mail.python.org/mailman/listinfo/python-list

Help on Dict

[Clement]

Can any body tell how Dict is implemented in python... plz tell what
datastructure that uses

-- 
http://mail.python.org/mailman/listinfo/python-list

Nested Parameter Definitions

[Paddy]

I blogged on finding a new-to-me feature of Python, in that you are
allowed to nnest parameter definitions:

>>> def x ((p0, p1), p2):
... return p0,p1,p2
...
>>> x(('Does', 'this'), 'work')
('Does', 'this', 'work')
>>>

Ruben commented that there was a poll on this features continued
existence taken at PyCon and it could go.

Just as I found it, it could go

I wondered if those of you with some Python experience new of nested
parameters and don't use them; or just forgot/don't know it is
possible?

- Paddy.

Oh - the blog entry is at 
http://paddy3118.blogspot.com/2007/02/pythons-function-nested-parameter.html

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Referencing Items in a List of Tuples

[rshepard]

On 2007-02-25, Dennis Lee Bieber <[EMAIL PROTECTED]> wrote:

>   Item is ALREADY the "current" tuple...
>
> for tpl in mainlist:
>   if tpl[0] == "eco" and tpl[1] == "con":
>   ec.Append(tpl[2:])  #presuming ec is NOT a list, as Append()
>   #is not a list method 
> (append() is)

  Of course! As a morning person I don't do as well at night. That's such a
silly error I'm embarrassed by having to have the obvious pointed out to me.
My apologies to all.

  And, the .Append comes from list widgets in wxPython.

  Thank you all for forcing my head straight again.

Rich
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: convert strings to utf-8

[Martin v. Löwis]

Niclas schrieb:
> I'm having trouble to work with the special charcters in swedish (Å Ä Ö
> å ä ö). The script is parsing and extracting information from a webpage.
> This works fine and I get all the data correctly. The information is
> then added to a rss file (using xml.dom.minidom.Document() to create the
> file), this is where it goes wrong. Letters like Å ä ö get messed up and
> the rss file does not validate. How can I convert the data to UTF-8
> without loosing the special letters?

You should convert the strings from the webpage to Unicode strings.
You can see that a string is unicode of

print isinstance(s,unicode)

prints True. Make sure *every* string you put into the Document
actually is a Unicode string. Then it will just work fine.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RegExp performance?

[Christian Sonne]

John Machin wrote:
> On Feb 25, 7:21 pm, Christian Sonne <[EMAIL PROTECTED]> wrote:
>> Long story short, I'm trying to find all ISBN-10 numbers in a multiline
>> string (approximately 10 pages of a normal book), and as far as I can
>> tell, the *correct* thing to match would be this:
>> ".*\D*(\d{10}|\d{9}X)\D*.*"
> 
> All of those *s are making it work too hard.
> 
> Starting with your r".*\D*(\d{10}|\d{9}X)\D*.*" [you do have the
> r"..." not just "", don't you?]
> 
> Step 1: Lose the .* off each end -- this is meaningless in the context
> of a search() or findall() and would slow the re engine down if it
> doesn't optimise it away.
> 
> r"\D*(\d{10}|\d{9}X)\D*"
> 
> Step 2: I presume that the \D* at each (remaining) end is to ensure
> that you don't pick up a number with 11 or more digits. You only need
> to test that your presumed ISBN is not preceded/followed by ONE
> suspect character. Is ABC1234567890DEF OK? I think not; I'd use \b
> instead of \D
> 
> r"\b(\d{10}|\d{9}X)\b"
> 
> Step 3: Now that we have only \b (which matches 0 characters) at each
> end of the ISBN, we can lose the capturing ()
> 
> r"\b\d{10}|\d{9}X\b"
> 
> Step 4: In the case of 123456789X, it fails on the X and then scans
> the 123456789 again -- a tiny waste compared to all the * stuff, but
> still worth fixing.
> 
> r"\b\d{9}[0-9X]\b"
> 
> Give that a whirl and let us know how correct and how fast it is.
> 
>> (it should be noted that I've removed all '-'s in the string, because
>> they have a tendency to be mixed into ISBN's)
>>
>> however, on my 3200+ amd64, running the following:
>>
>> reISBN10 = re.compile(".*\D*(\d{10}|\d{9}X)\D*.*")
> 
> You should really get into the habit of using raw strings with re.
> 
>> isbn10s = reISBN10.findall(contents)
>>
>> (where contents is the string)
>>
>> this takes about 14 minutes - and there are only one or two matches...
> 
> How many actual matches and how many expected matches?
> 
> Note on "and there are only one or two matches": if your data
> consisted only of valid ISBNs separated by a single space or comma, it
> would run much faster. It is all the quadratic/exponential mucking
> about with the in-between bits that slows it down. To demonstrate
> this, try timing dummy data like "1234567890 " * 100 and
> "123456789X " * 100 with your various regexes, and with the step1,
> step2 etc regexes above.
> 
>> if I change this to match ".*[ ]*(\d{10}|\d{9}X)[ ]*.*" instead, I risk
>> loosing results, but it runs in about 0.3 seconds
>>
>> So what's the deal? - why would it take so long to run the correct one?
> 
> Because you have .*\D* in other words 0 or more occurrences of almost
> anything followed by 0 or more occurrences of almost anything. Even
> assuming it ignores the .*, it will find the longest possible sequence
> of non-digits, then try to match the ISBN stuff. If it fails, it will
> shorten that sequence of non-digits, try the ISBN stuff again, etc etc
> until it matches the ISBN stuff or that sequence of non-digits is down
> to zero length. It will do that for each character position in the
> file contents. Why is it faster when you change \D to []? Presumably
> because in your data, sequences of non-digits are longer than
> sequences of spaces IOW there is less stuff to back-track over.
> 
> 
>> - especially when a slight modification makes it run as fast as I'd
>> expect from the beginning...
>>
>> I'm sorry I cannot supply test data, in my case, it comes from
>> copyrighted material - however if it proves needed, I can probably
>> construct dummy data to illustrate the problem
> 
> What you call "dummy data", I'd call "test data". You should make a
> sample of "dummy data" and test that your regex is (a) finding all
> ISBNs that it should (b) not reporting incorrect matches, *BEFORE*
> being concerned with speed.
> 
>> Any and all guidance would be greatly appreciated,
> 
> For some light reading :-) borrow a copy of Friedl's book (mentioned
> in the Python re docs) and read the parts on how backtracking regex
> engines work.
> 
> HTH,
> John
> 

Thanks to all of you for your replies - they have been most helpful, and 
my program is now running at a reasonable pace...


I ended up using r"\b\d{9}[0-9X]\b" which seems to do the trick - if it 
turns out to misbehave in further testing, I'll know where to turn :-P
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to build Hierarchies of dict's? (Prototypes in Python?)

[Charles D Hixson]

Toby wrote:
> Charles D Hixson wrote:
>   
>> a class whose sub-classes automatically have unique class variables of
>> a determined form such that I can do a hierarchical search through them
>> 
>
> Something like this?  
> (scroll down to see the results)
>
>
> # --- begin ---
>
> class AttrSearch(object):
>   @classmethod
>   def getclassattrset(cls, name):
> s = set()
> if name in cls.__dict__:
>   s.add((cls.__name__, cls.__dict__[name]))
> for base in cls.__bases__:
>   try:
> s.update(base.getclassattrset(name))
>   except AttributeError:
> pass
> return s
>
>   def getattrset(self, name):
> s = set()
> try:
>   s.add((None, self.__dict__[name]))
> except KeyError:
>   pass
> s.update(self.__class__.getclassattrset(name))
> return s
>
>   def __getattribute__(self, name):
> if name.startswith('__'): #XXX not pretty
>   return object.__getattribute__(self, name)
> found = AttrSearch.getattrset(self, name)
> print 'Looking for "%s" in a %s instance, found %d candidates:' \
>   % (name, self.__class__.__name__, len(found))
> print '\n'.join([ '  %-4s %s' % x for x in found ])
> print '(now choose wisely what to return)'
>
> class A(AttrSearch):
>   a = 1
>
> class B(A):
>   a = 2
>
> class C(A):
>   a = 3
>
> class D(B, C):
>   a = 4
>
>
> D().a
>
> # --- end ---
>
>
> Results:
>
> Looking for "a" in a D instance, found 4 candidates:
>   A1
>   B2
>   C3
>   D4
> (now choose wisely what to return)
>
>
> Toby
>   
Yes, thank you.
(I'd come up with a kludge, but this is much nicer.)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[Virgil Dupras]

On Feb 25, 1:00 pm, "Paddy" <[EMAIL PROTECTED]> wrote:
> I blogged on finding a new-to-me feature of Python, in that you are
> allowed to nnest parameter definitions:
>
> >>> def x ((p0, p1), p2):
>
> ... return p0,p1,p2
> ...>>> x(('Does', 'this'), 'work')
>
> ('Does', 'this', 'work')
>
>
>
> Ruben commented that there was a poll on this features continued
> existence taken at PyCon and it could go.
>
> Just as I found it, it could go
>
> I wondered if those of you with some Python experience new of nested
> parameters and don't use them; or just forgot/don't know it is
> possible?
>
> - Paddy.
>
> Oh - the blog entry is 
> athttp://paddy3118.blogspot.com/2007/02/pythons-function-nested-paramet...

I didn't know about it either. Without the call example, I would have
had a hard time to try to figure out what these extra brackets are
for. For this reason, I think that an explicit unpack is more
readable, and thus better.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rational numbers

[aleaxit]

On Feb 24, 12:25 am, Toby A Inkster <[EMAIL PROTECTED]>
wrote:
> aleaxit wrote:
> > If anybody who has easy access to Microsoft's MSVC++.NET (and is willing
> > to try building GMP 4.2 with/for it), or a PPC Mac with XCode installed
> > (possibly with MacOSX 10.3...)
>
> I'm writing this message on a MacOS 10.3.9 box with Xcode 1.5 (gcc 3.3)
> installed. If you tell me how, I'd be happy to compile it for you.
>
> Contact me through the feedback form on the site below.

Thanks, but unfortunately getting GMP 4.2 built right on any platform
is never a trivial task -- GMP's maintainer is apparently actively
hostile to any attempts of making fixes to GMP to make it build
correctly on MAC OS X, in particular.  I have a correspondent who
thinks he's got a working patch (for PPC on MacOSX 10.4, at least);
he's very busy but I hope he'll send it to me eventually.

gmpy itself is or should be pretty trivial to build on any platform
(and I'll always happily accept any fixes that make it better on any
specific platform, since it's easy to make them conditional so they'll
apply to that platform only), but the underlying GMP is anything but:-
(.


Alex

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: finding out the precision of floats

[Robert Kern]

John Machin wrote:

> Evidently not; here's some documentation we both need(ed) to read:
> 
> http://docs.python.org/tut/node16.html
> """
> Almost all machines today (November 2000) use IEEE-754 floating point
> arithmetic, and almost all platforms map Python floats to IEEE-754
> "double precision".
> """
> I'm very curious to know what the exceptions were in November 2000 and
> if they still exist.

All Python interpreters use whatever is the C double type on its platform to
represent floats. Not all of the platforms use *IEEE-754* floating point types.
The most notable and still relevant example that I know of is Cray.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Who has not attended these free tutorial courses ?

[*]

Is it true that all the above-listed "tutorials" offer undergraduate credit
at Islam University?


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on Dict

[James Stroud]

Clement wrote:
> Can any body tell how Dict is implemented in python... plz tell what
> datastructure that uses
> 

I think it uses a dict.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[Arnaud Delobelle]

On Feb 25, 6:00 pm, "Paddy" <[EMAIL PROTECTED]> wrote:
> I blogged on finding a new-to-me feature of Python, in that you are
> allowed to nnest parameter definitions:
>
> >>> def x ((p0, p1), p2):
>
> ... return p0,p1,p2
> ...>>> x(('Does', 'this'), 'work')
>
> ('Does', 'this', 'work')

Reminds me of LeLisp! It had a similar feature.  IIRC you could write
for example (I think 'df' was LeLisp for 'defun'):
(df mycar (a . b) a)
or
(df mylist L L)
or
(df mycaadr (a (b . c) . e) b)

I didn't know that this was possible in python and it does surprise
me.  It feels at odd with the python philosophy.

--
Arnaud

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about idiomatic use of _ and private stuff.

[Bruno Desthuilliers]

Steven W. Orr a écrit :
> I understand that two leading underscores in a class attribute make the 
> attribute private. 

Nope. It doesn't make it "private", it mangles the attribute name with 
the class name (ie : Bar.__mangled will become Bar._Bar__mangled 
everywhere except inside Bar). This is only useful when you want to make 
sure an attribute will not be *accidentally* accessed by a child class. 
FWIW, I've found it of very limited use so far...

> But I often see things that are coded up with one 
> underscore. Unless I'm missing something, there's a idiom going on here.

Yes. Single leading underscore means "this is implementation, don't mess 
with it or your on your own". It's the equivalent of "private".

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Referencing Items in a List of Tuples

[Paddy]

On Feb 25, 5:44 pm, Jussi Salmela <[EMAIL PROTECTED]> wrote:
> Paddy kirjoitti:
>
>
>
> > On Feb 25, 2:01 am, [EMAIL PROTECTED] wrote:
> >>   While working with lists of tuples is probably very common, none of my
> >> five Python books or a Google search tell me how to refer to specific items
> >> in each tuple. I find references to sorting a list of tuples, but not
> >> extracting tuples based on their content.
>
> >>   In my case, I have a list of 9 tuples. Each tuple has 30 items. The first
> >> two items are 3-character strings, the remaining 28 itmes are floats.
>
> >>   I want to create a new list from each tuple. But, I want the selection of
> >> tuples, and their assignment to the new list, to be based on the values of
> >> the first two items in each tuple.
>
> >>   If I try, for example, writing:
>
> >> for item in mainlist:
> >> if mainlist[item][0] == 'eco' and mainlist[item][1] == 'con':
> >>   ec.Append(mainlist[item][2:])
>
> >> python doesn't like a non-numeric index.
>
> >>   I would really appreciate a pointer so I can learn how to manipulate 
> >> lists
> >> of tuples by referencing specific items in each tuple (string or float).
>
> >> Rich
>
> > You might also use list comprehensions to accumulate the values you
> > need:
>
> > ec = [ item[2:]  for item in mainlist  if item[:2] == ['eco','con'] ]
>
> > - Paddy.
>
> I'm nitpicking, but the OP has a list of tuples:
>
> ec = [ item[2:]  for item in mainlist  if item[:2] == ('eco','con') ]
>
> Cheers,
> Jussi

Of course. Ta.
- paddy.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about idiomatic use of _ and private stuff.

[Bruno Desthuilliers]

Troy Melhase a écrit :
>> Why do people sometimes use one leading underscore?
> 
> 
> Many folks like to use the single leading underscore to emphasize that
> the attribute isn't part of the normal way to use the class or
> instance.
> 
> It's bad style in my opinion, but I'm probably in the minority.

You are, definitively. While it's not technically part of the language 
syntax, you can practically consider it as such.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on Dict

[Daniel Nogradi]

> Can any body tell how Dict is implemented in python... plz tell what
> datastructure that uses

Please see

http://svn.python.org/view/python/trunk/Objects/dictnotes.txt?rev=53782&view=markup

for general notes and

http://svn.python.org/view/python/trunk/Objects/dictobject.c?rev=53911&view=markup

for the actual implementation.

HTH,
Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Referencing Items in a List of Tuples

[Bruno Desthuilliers]

[EMAIL PROTECTED] a écrit :
>   While working with lists of tuples is probably very common, none of my
> five Python books or a Google search tell me how to refer to specific items
> in each tuple.

t = "a", "b", "c"
assert t[0] == "a"
assert t[1] == "b"
assert t[2] == "c"


> I find references to sorting a list of tuples, but not
> extracting tuples based on their content.

loft = [(1, 2, 3), (1, 2, 4), (2, 3, 4)]
starting_with_one = [t for t in loft if t[0] == 1]

>   In my case, I have a list of 9 tuples. Each tuple has 30 items. The first
> two items are 3-character strings, the remaining 28 itmes are floats.
> 
>   I want to create a new list from each tuple. But, I want the selection of
> tuples, and their assignment to the new list, to be based on the values of
> the first two items in each tuple.
> 
>   If I try, for example, writing:
> 
> for item in mainlist:
> if mainlist[item][0] == 'eco' and mainlist[item][1] == 'con':
>   ec.Append(mainlist[item][2:])
> 
> python doesn't like a non-numeric index.

Hem... I'm afraid you don't understand Python's for loops. The above 
should be:

ec = []
for item in mainlist:
 if [item][0] == 'eco' and item[1] == 'con':
 ec.append(item[2:])

which can also be written:
ec = [item[2:] for item in mainList \
   if item[0] == 'eco' and item[1] == 'con']

>   I would really appreciate a pointer so I can learn how to manipulate lists
> of tuples by referencing specific items in each tuple (string or float).

Did you actually tried reading some Python 101 material ?

-- 
http://mail.python.org/mailman/listinfo/python-list

BeautifulSoup modified to use weak refs and avoid circular links.

[John Nagle]

John Nagle wrote:
>   Are weak refs slower than strong refs?  I've been considering making the
> "parent" links in BeautifulSoup into weak refs, so the trees will release
> immediately when they're no longer needed.  In general, all links back
> towards the root of a tree should be weak refs; this breaks the loops
> that give reference counting trouble.
> 
> John Nagle

I just finished converting BeautifulSoup to use weak back references.
All that's necessary is to make the "parent", "previous", "previousSibling",
and "tagStack" links into weak proxy references.  Those are the
"backlinks" of the Beautiful Soup tree; once they're weak links,
BeautifulSoup then becomes loop-free and GC in debug mode reports
zero garbage.  This is useful, because BeautifulSoup's backlinks create huge 
amounts of collectable garbage, leading to frequent GC cycles.

The "weakref" module could use some support functions.  If you
try to create a weakref proxy from a weakref proxy, or from "None",
you get an exception, which is not usually what you want.
It's more useful to pass through "None" or an existing weakref proxy.
So I wrote the function below, which makes it much easier to
convert code to use weak proxy references.

"weakref.proxy()" probably should work that way.
Weakref proxies are supposed to be transparent, but they're not
quite transparent enough.

Patches to BeautifulSoup are below.

John Nagle

59a60,80
 > import weakref   # Weak 
 > references for previous, parent, previousSibling
 > #
 > #Weakref allocation control
 > #
 > #The following links are always weak references, to avoid internal 
 > referenc
 > #require extra garbage collection.
 > #self.parent
 > #self.previous
 > #self.previousSibling
 > #These are all "back links".
 > #
 > #backref  --  create a weak reference as a back pointer
 > #
 > #Generates a weakref proxy, but handles input of "none" or an existing 
 > weak
 > #
 > def backref(p) :
 >  if p == None :  
 > # if none
 >  return(None)
 > # then none
 >  if isinstance(p,weakref.ProxyType) or 
 > isinstance(p,weakref.CallableProxyTyp
 >  return(p)
 >  return(weakref.proxy(p))
 > # otherwise a new weakref
60a82
 > #
79,80c101,102
< self.parent = parent
< self.previous = previous
---
 > self.parent = backref(parent)
 > self.previous = backref(previous)
85c107
< self.previousSibling = self.parent.contents[-1]
---
 > self.previousSibling = backref(self.parent.contents[-1])
127c149
< self.nextSibling.previousSibling = self.previousSibling
---
 > self.nextSibling.previousSibling = backref(self.previousSibling)
157c179
< newChild.parent = self
---
 > newChild.parent = backref(self)
161c183
< newChild.previous = self
---
 > newChild.previous = backref(self)
164c186
< newChild.previousSibling = previousChild
---
 > newChild.previousSibling = backref(previousChild)
166c188
< newChild.previous = previousChild._lastRecursiveChild()
---
 > newChild.previous = backref(previousChild._lastRecursiveChild())
190c212
< newChild.nextSibling.previousSibling = newChild
---
 > newChild.nextSibling.previousSibling = backref(newChild)
194c216
< newChildsLastElement.next.previous = newChildsLastElement
---
 > newChildsLastElement.next.previous = backref(newChildsLastElemen
1052c1074
< self.tagStack.append(tag)
---
 > self.tagStack.append(backref(tag))
1074c1096
< self.previous = o
---
 > self.previous = backref(o)
1167c1189
< self.previous = tag
---
 > self.previous = backref(tag)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[Paddy]

On Feb 25, 7:06 pm, "Virgil Dupras" <[EMAIL PROTECTED]>
wrote:
> On Feb 25, 1:00 pm, "Paddy" <[EMAIL PROTECTED]> wrote:
>
>
>
> > I blogged on finding a new-to-me feature of Python, in that you are
> > allowed to nnest parameter definitions:
>
> > >>> def x ((p0, p1), p2):
>
> > ... return p0,p1,p2
> > ...>>> x(('Does', 'this'), 'work')
>
> > ('Does', 'this', 'work')
>
> > Ruben commented that there was a poll on this features continued
> > existence taken at PyCon and it could go.
>
> > Just as I found it, it could go
>
> > I wondered if those of you with some Python experience new of nested
> > parameters and don't use them; or just forgot/don't know it is
> > possible?
>
> > - Paddy.
>
> > Oh - the blog entry is 
> > athttp://paddy3118.blogspot.com/2007/02/pythons-function-nested-paramet...
>
> I didn't know about it either. Without the call example, I would have
> had a hard time to try to figure out what these extra brackets are
> for. For this reason, I think that an explicit unpack is more
> readable, and thus better.

The following example shows three possible ways of accessing nested
data. i think , after now knowing how to use it, that the f0 function
with nested parameters and its subsequent use is the most readable.
Manual unpacking in the list comprehension for f1 is messy. No real
reason for liking f0 over f2 except its shiny!

>>> def f0 ((p0, p1), p2):
... pass
...
>>> def f1 (p0, p1, p2):
... pass
...
>>> def f2(p):
... (p0, p1), p2 = p
...
>>> data = [ ((1, 2), 3),  ((4, 5), 6)]
>>> [ f0(*datum) for datum in data]
[None, None]
>>> [ f1(datum[0][0], datum[0][1], datum[1]) for datum in data]
[None, None]
>>> [ f2(datum) for datum in data]
[None, None]
>>>

- Paddy.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to build Hierarchies of dict's? (Prototypes in Python?)

[Toby]

Charles D Hixson wrote:
> I want to access the values via instances of the class, so your D().a
> approach isn't suitable.  More like:  "t = D(); t[a]"

Well, D() is an instance of D, so  D().a  is the same as  t = D(); t.a
In fact the various "a" attributes I'm accessing are class attributes of
the various classes, just as you asked.

If you'd rather use t[a], then things are simpler, because you won't
need to override __getattribute__, with all the ugliness that comes with
it: __getitem__ will suffice.


> Your approach uses techniques that I'm going to need to study
> carefully before I can hope to understand them.

Don't worry, here's a heavily commented version, just because it's
sunday and I have some free time ;-)


# inheriting from object so that we can use __getattribute__
# see http://www.python.org/download/releases/2.2.3/descrintro/
class AttrSearch(object):  
  '''Base class that customizes attribute access in all of its derived
  classes, choosing between all the candidates in a custom way.'''


  @classmethod
  def getclassattrset(cls, name):
'''Class method (= polymorphic static method) that collects a set
of the attributes with a given name from itself and from all its
base classes (determined at runtime.)

The only argument is the name of the attribute to look up; the
return value is a list of (class_name, attribute_value) listing all
the candidates found.'''

# note: the first parameter 'cls' is the class of the runtime object
# on which this class method is called, much like 'self' is the
# object instance in instance methods
#
# notice: this method is defined on class AttrSearch, but if I call
# it on B (subclass of AttrSearch), then 'cls' is class B!

s = set()  # a set is an unordered list without duplicates

try:
  # __dict__ is the internal dictionary which holds all of a class
  # or of an instance's attributes

  # creating a tuple with the name of the runtime class 'cls'
  # and the value of cls's attribute named name, and adding the
  # tuple to the set (unless an equal tuple is already there)
  s.add((cls.__name__, cls.__dict__[name]))
except KeyError:
  # ...unless the class cls has no such attribute
  pass

# for every base class, from which the runtime class inherits:
for base in cls.__bases__:
  try:
# call this same method on them and add their results to the set
s.update(base.getclassattrset(name))
  except AttributeError:
# except for the base classes which don't implement this method
# (for example the class object)
pass

return s  # returning the collected set


  def getattrset(self, name):
'''Instance method that collects a set of the attributes with a
given name from itself, from its class and from all the base classes.

The only argument is the name of the attribute to look up; the
return value is a list of (class_name, attribute_value) listing all
the candidates found.  In case the attribute is also found in the
object itself, element 0 of the tuple is set to null.'''

# saving references to a couple of attributes we need to access
# directly, bypassing all this machinery; to achieve it, we
# explicitly call object's implementation of __getattribute__

self_dict = object.__getattribute__(self, '__dict__')
self_class = object.__getattribute__(self, '__class__')

s = set()

try:
  # adding to the set the attribute named name in this very intance,
  # if it exists, with None in place of the class name
  s.add((None, self_dict[name]))
except KeyError:
  # unless the instance doesn't have an attribute named name
  pass

# addig to the set all the class attributes named name, from this
# object's class and all its base classes
s.update(self_class.getclassattrset(name))

return s


  def __getattribute__(self, name):
'''Customized version of attribute fetching, that uses getattrset
(and thus getclassattrset) to get a list of all the attributes named
name before choosing which one to return.'''

# saving references to the attributes we need to access directly
self_class = object.__getattribute__(self, '__class__')

# calling AttrSearch's getattrset to do the dirty work
found = AttrSearch.getattrset(self, name)

# here is where you should examine 'found' and choose what to
# return; I only print what is available and return None
print 'Looking for "%s" in a %s instance, found %d candidates:' \
% (name, self_class.__name__, len(found))
print '  class  value'
print '  =  ='
print '\n'.join([ "  %-6s '%s'" % x for x in found ])
print '(now choose wisely what to return)'

return None


# example use of AttrSearch in a class hierarchy:

class A(AttrSearch):
  a = 'attribute "a" of class A'

class B(A):
  a = 'attribute "a" of class B'

class C(A):
  a = 

Help on object scope?

[bmaron2]

Hello everybody,

I have a (hopefully) simple question about scoping in python. I have a
program written as a package, with two files of interest. The two
files are /p.py and /lib/q.py

My file p.py looks like this:

---

from lib import q

def main():
  global r
  r = q.object1()
  s = q.object2()

if __name__ == "__main__":
  main()

---

My file q.py in the subdirectory lib looks like this:

class object1:
  t = 3

class object2:
  print r.t

---

Python gives me an error, saying it can't recognize global name r.
However I define r as global in the top-level main definition! Can
anyone suggest how I can get around this, if I want to define and bind
global names inside of main() which are valid in all sub-modules?

Thanks very much for your help!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on object scope?

[hg]

[EMAIL PROTECTED] wrote:

> Hello everybody,
> 
> I have a (hopefully) simple question about scoping in python. I have a
> program written as a package, with two files of interest. The two
> files are /p.py and /lib/q.py
> 
> My file p.py looks like this:
> 
> ---
> 
> from lib import q
> 
> def main():
>   global r
>   r = q.object1()
>   s = q.object2()
> 
> if __name__ == "__main__":
>   main()
> 
> ---
> 
> My file q.py in the subdirectory lib looks like this:
> 
> class object1:
>   t = 3
> 
> class object2:
>   print r.t
> 
> ---
> 
> Python gives me an error, saying it can't recognize global name r.
> However I define r as global in the top-level main definition! Can
> anyone suggest how I can get around this, if I want to define and bind
> global names inside of main() which are valid in all sub-modules?
> 
> Thanks very much for your help!

Might be wrong, but globals can only be global to the module they're
declared in.

I suggest you find another way such as passing your object as a parameter

hg


-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: pynakotheka v1.1.0

[Iñigo Serna]

Hi there,

I'm pleased to announce a new release of Pynakotheka.

Pynakotheka is a simple GPL-licensed python script which generates
static HTML photo albums to be added to web sites or to be burnt into
CDs.
It includes some templates and it's easy to create more.

It depends on python, Mako Templates, EXIF and PIL.

Read more and download it from:

http://inigo.katxi.org/devel/pynakotheka

or http://www.terra.es/personal7/inigoserna/pynakotheka


Changes from v1.0.3 to v1.1.0:
==
* use Mako Templates instead of Cheetah Templating System:
  - html creation is much faster now
  - old problems with text encodings should be solved now
* file names encoded in UTF8 works ok now, or so I hope
* added a "--version" option

As always, all comments, suggestions etc. are welcome.


Best regards,
Iñigo Serna

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: convert strings to utf-8

[Niclas]

Thank you!

solved it with this:
  unicode( data.decode('latin_1') )
and when I write it to the file...
 f = codecs.open(path, encoding='utf-8', mode='w+')
 f.write(self.__rssDoc.toxml())

Diez B. Roggisch skrev:
> Niclas schrieb:
>> Hi
>>
>> I'm having trouble to work with the special charcters in swedish (Å Ä 
>> Ö å ä ö). The script is parsing and extracting information from a 
>> webpage. This works fine and I get all the data correctly. The 
>> information is then added to a rss file (using 
>> xml.dom.minidom.Document() to create the file), this is where it goes 
>> wrong. Letters like Å ä ö get messed up and the rss file does not 
>> validate. How can I convert the data to UTF-8 without loosing the 
>> special letters?
> 
> Show us code, and example text (albeit I know it is difficult to get 
> that right using news/mail)
> 
> The basic idea is this:
> 
> scrapped_byte_string = scrap_the_website()
> 
> output = scrappend_byte_string.decode('website-encoding').encode('utf-8')
> 
> 
> 
> Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on object scope?

[bmaron2]

On Feb 25, 9:37 am, hg <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote:
> > Hello everybody,
>
> > I have a (hopefully) simple question about scoping in python. I have a
> > program written as a package, with two files of interest. The two
> > files are /p.py and /lib/q.py
>
> > My file p.py looks like this:
>
> > ---
>
> > from lib import q
>
> > def main():
> >   global r
> >   r = q.object1()
> >   s = q.object2()
>
> > if __name__ == "__main__":
> >   main()
>
> > ---
>
> > My file q.py in the subdirectory lib looks like this:
>
> > class object1:
> >   t = 3
>
> > class object2:
> >   print r.t
>
> > ---
>
> > Python gives me an error, saying it can't recognize global name r.
> > However I define r as global in the top-level main definition! Can
> > anyone suggest how I can get around this, if I want to define and bind
> > global names inside of main() which are valid in all sub-modules?
>
> > Thanks very much for your help!
>
> Might be wrong, but globals can only be global to the module they're
> declared in.
>
> I suggest you find another way such as passing your object as a parameter
>
> hg

Dear hg,

Thank you for the advice, but that seems a bit unwieldy. My code will
have about 10 of these global objects, all of which interact with
eachother. It seems silly to have to pass 10 parameters around to each
instance I work with. I hope there is a smarter way to do it, or
perhaps someone can suggest a smarter way to code it.

Am I stuck with just having to put all the code in one file? That is
what I wanted to avoid, because the file will get incredibly long. It
seems like the only reason my code above failed is because there were
two separate modules. Perhaps I just need to import /lib/q.py in a
different way?

Many thanks.

-- 
http://mail.python.org/mailman/listinfo/python-list

Weakref problem: no way to get original object from proxy object?

[John Nagle]

Is there some way to get a strong ref to the original object back
from a weakref proxy object?  I can't find any Python function to do this.
".ref()" doesn't work on proxy objects.

John Nagle
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: convert strings to utf-8

[Diez B. Roggisch]

Niclas schrieb:
> Thank you!
> 
> solved it with this:
>  unicode( data.decode('latin_1') )

The unicode around this is superfluous. Either do

unicode(bytestring, encoding)

or

bytestring.decode(encoding)


> and when I write it to the file...
> f = codecs.open(path, encoding='utf-8', mode='w+')
> f.write(self.__rssDoc.toxml())


Looks good, yes.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on object scope?

[Ben Finney]

[EMAIL PROTECTED] writes:

> My code will have about 10 of these global objects, all of which
> interact with eachother. It seems silly to have to pass 10
> parameters around to each instance I work with.

Yes. A better solution would be to put them inside a module, and
import that module. Then the objects are available at any level as
attributes of the imported module.

= foo.py =
def spam():
return max(eggs, ham)

eggs = 17
ham = 12
=

= bar.py =
import foo

def main():
spork = do_something_with(foo.eggs)
foo.spam(spork)
=

-- 
 \"Know what I hate most? Rhetorical questions."  -- Henry N. Camp |
  `\   |
_o__)  |
Ben Finney

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on object scope?

[Diez B. Roggisch]

> 
> Thank you for the advice, but that seems a bit unwieldy. My code will
> have about 10 of these global objects, all of which interact with
> eachother. It seems silly to have to pass 10 parameters around to each
> instance I work with. I hope there is a smarter way to do it, or
> perhaps someone can suggest a smarter way to code it.
> 
> Am I stuck with just having to put all the code in one file? That is
> what I wanted to avoid, because the file will get incredibly long. It
> seems like the only reason my code above failed is because there were
> two separate modules. Perhaps I just need to import /lib/q.py in a
> different way?

You can interact just fine, just qualify the objects with the module 
names. So in q, you need to use p.r instead of just r.

It's another question if this design is really good - relying on so many 
globals. It would certainly be better to have e.g. a class that looks 
like this:


import p
import q
class GlueThingy(object):
 def __init__(self):
 self.p1 = p.SomeObject(self)
 self.q1 = q.SomeOtherObject(self)


Then in SomeObject you can use the passed GlueThingy reference to access 
other instances:

class SomeObject(object):

 def __init__(self, glue):
 self.glue = glue

 def do_something(self):
 self.glue.q1.foobar()


Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: BeautifulSoup modified to use weak refs and avoid circular links.

[Delaney, Timothy (Tim)]

John Nagle wrote:

> "weakref.proxy()" probably should work that way.
> Weakref proxies are supposed to be transparent, but they're not
> quite transparent enough.

Submit a patch to SourceForge. Please don't use tabs in email/usenet
postings - use 4-space indents. "return" is not a function, and
comparisons with "None" should nearly always use "is".

I've also changed backref() to use isinstance(obj, tuple) (available in
Python 2.2 and later):

def backref(p):
if p is None:
return None

if isinstance(p, (weakref.ProxyType,
weakref.CallableProxyType)):
return p

return weakref.proxy(p)

Tim Delaney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rational numbers

[Fernando Perez]

[EMAIL PROTECTED] wrote:


> gmpy itself is or should be pretty trivial to build on any platform
> (and I'll always happily accept any fixes that make it better on any
> specific platform, since it's easy to make them conditional so they'll
> apply to that platform only), but the underlying GMP is anything but:-
> (.

Alex, have you had a look at SAGE?

http://modular.math.washington.edu/sage/

it uses GMP extensively, so they've had to patch it to work around these
issues.  You can look at the SAGE release (they package everything as the
original tarball + patches) for the GMP-specific stuff you need, though I
suspect you'll still want to play with SAGE a little bit :).  It's a mighty
impressive system.

Cheers,


f

-- 
http://mail.python.org/mailman/listinfo/python-list

Working with C object of python object (pycairo,ctypes)

[AngelBlaZe]

Can you access the c object of a python object directly? Can this be
done in ctypes or directly throught python functions without reverting
to forking the python module?

scanario:
I have pyctx object that i get in python like this:

pyctx = cairo.Context(surface)

defined in the python extension this:

typedef struct {
PyObject_HEAD
cairo_t *ctx;
PyObject *base; /* base object used to create context, or NULL */
} PycairoContext;

PyObject *
PycairoContext_FromContext(cairo_t *ctx, PyTypeObject *type, PyObject
*base)
{
PyObject *o;

assert (ctx != NULL);

if (Pycairo_Check_Status (cairo_status (ctx))) {
cairo_destroy (ctx);
return NULL;
}

if (type == NULL)
type = &PycairoContext_Type;
o = PycairoContext_Type.tp_alloc (type, 0);
if (o) {
((PycairoContext *)o)->ctx = ctx;
Py_XINCREF(base);
((PycairoContext *)o)->base = base;
} else {
cairo_destroy (ctx);
}
return o;
}

Now my question is:
Is there any way to access/get a pointer to PycairoContext->ctx from
python object (pyctx)?

A long shot i know :-)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Referencing Items in a List of Tuples

[rshepard]

On 2007-02-25, Paddy <[EMAIL PROTECTED]> wrote:

> You might also use list comprehensions to accumulate the values you
> need:
>
> ec = [ item[2:]  for item in mainlist  if item[:2] == ['eco','con'] ]

  Thank you, Paddy. That's the syntax I couldn't work out myself.

Rich
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Referencing Items in a List of Tuples

[rshepard]

On 2007-02-25, Jussi Salmela <[EMAIL PROTECTED]> wrote:

> I'm nitpicking, but the OP has a list of tuples:
> ec = [ item[2:]  for item in mainlist  if item[:2] == ('eco','con') ]

Jussi,

  An excellent nit to pick.

Thank you,

Rich
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[Steven D'Aprano]

On Sun, 25 Feb 2007 10:00:31 -0800, Paddy wrote:

> I wondered if those of you with some Python experience new of nested
> parameters and don't use them; or just forgot/don't know it is
> possible?

I learnt about this some time ago. I don't often use it, although it makes
sense to write this:

def parrot(x, (y, z)):
pass

instead of this:

def parrot(x, a2tuple):
y, z = a2tuple
pass



-- 
Steven D'Aprano 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[Steven D'Aprano]

On Sun, 25 Feb 2007 11:06:03 -0800, Virgil Dupras wrote:

> On Feb 25, 1:00 pm, "Paddy" <[EMAIL PROTECTED]> wrote:
>> I blogged on finding a new-to-me feature of Python, in that you are
>> allowed to nnest parameter definitions:
>>
>> >>> def x ((p0, p1), p2):
>>
>> ... return p0,p1,p2

[snip]

> I didn't know about it either. Without the call example, I would have
> had a hard time to try to figure out what these extra brackets are
> for. For this reason, I think that an explicit unpack is more
> readable, and thus better.

And now that you do know, it is not hard to figure it out at all.

Nested parameters are no harder to figure out than *args or arg=value.
It's something that you learn, once, and then it is easy forever.

To my mind, def x ((p0, p1), p2) _is_ an explicit unpack. It just takes
place in the parameter definition, where you can see it even if the
source code is not available, not in the body of the function code, where
you may or may not even be able to find it.



-- 
Steven D'Aprano 

-- 
http://mail.python.org/mailman/listinfo/python-list

newbie question

[S.Mohideen]

>>> asd={}
>>> asd={1:2,3:4,4:5}
>>> print asd
{1: 2, 3: 4, 4: 5}

>>> asd.has_key(3)
True
>>> asd.update()
>>> print asd
{1: 2, 3: 4, 4: 5}
>>>

what does asd.update() signifies. What is the use of it. Any comments would 
help to understand it.

Thanks
Mohideen 

-- 
http://mail.python.org/mailman/listinfo/python-list

getting terminal display size?

[jeff]

I looked around a lot on the internet and couldn't find out how to do
this, how do I get the sizer (in rows and columns) of the view?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question

[jeff]

On Feb 25, 7:49 pm, "S.Mohideen" <[EMAIL PROTECTED]>
wrote:
>...
>
> what does asd.update() signifies. What is the use of it. Any comments would
> help to understand it.
>
> Thanks
> Mohideen

>>> print {}.update.__doc__
D.update(E, **F) -> None.  Update D from E and F: for k in E: D[k] =
E[k]
(if E has keys else: for (k, v) in E: D[k] = v) then: for k in F: D[k]
= F[k]
so basically, this is what it does:

>>> e = {}
>>> f = {}
>>> f['a'] = 123
>>> e.update(f)
>>> e
{'a': 123}

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question

[Steven D'Aprano]

On Sun, 25 Feb 2007 18:49:56 -0600, S.Mohideen wrote:

 asd={}
 asd={1:2,3:4,4:5}

You don't need to initialise asd to an empty dict.


 print asd
> {1: 2, 3: 4, 4: 5}
> 
 asd.has_key(3)
> True
 asd.update()
 print asd
> {1: 2, 3: 4, 4: 5}

You're not updating it with anything.


> what does asd.update() signifies. What is the use of it. Any comments
> would help to understand it.

Try this:

>>> asd = {1: 2, 3: 4, 4: 5}
>>> D = {1: -99, 7: -99}
>>> asd.update(D)
>>> asd
{1: -99, 3: 4, 4: 5, 7: -99}



-- 
Steven D'Aprano 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question

[Steven D'Aprano]

On Sun, 25 Feb 2007 16:57:25 -0800, jeff wrote:

 print {}.update.__doc__

That's a long way to spell "help({}.update)" 

:-)


-- 
Steven D'Aprano 

-- 
http://mail.python.org/mailman/listinfo/python-list

modifying a list while iterating through

[dustin . getz]

consider the following working loop where Packet is a subclass of
list, with Packet.insert(index, iterable) inserting each item in
iterable into Packet at consecutive indexes starting at index.

i=0
while(ihttp://mail.python.org/mailman/listinfo/python-list

Re: finding out the precision of floats

[Robert Kern]

Dennis Lee Bieber wrote:
> On 25 Feb 2007 05:31:11 -0800, "John Machin" <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
> 
>> Evidently not; here's some documentation we both need(ed) to read:
>>
>> http://docs.python.org/tut/node16.html
>> """
>> Almost all machines today (November 2000) use IEEE-754 floating point
>> arithmetic, and almost all platforms map Python floats to IEEE-754
>> "double precision".
>> """
>> I'm very curious to know what the exceptions were in November 2000 and
>> if they still exist. There is also the question of how much it matters
> 
>   Maybe a few old Vaxes/Alphas running OpenVMS... Those machines had
> something like four or five different floating point representations (F,
> D, G, and H, that I recall -- single, double, double with extended
> exponent range, and quad)

I actually used Python on an Alpha running OpenVMS a few years ago. IIRC, the
interpreter was built with IEEE floating point types rather than the other 
types.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[Gabriel Genellina]

En Sun, 25 Feb 2007 15:00:31 -0300, Paddy <[EMAIL PROTECTED]>  
escribió:

 def x ((p0, p1), p2):
> ... return p0,p1,p2

The first time I saw it used was in Zope, a long time ago. And I like it.  
Of course it only has any sense if you expect the tuple (p0,p1) to exist  
*before* the function is called; by example, when it's the return value of  
some other function.

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: convert strings to utf-8

[John Nagle]

Diez B. Roggisch wrote:
> Niclas schrieb:
> 
>> Thank you!
>>
>> solved it with this:
>>  unicode( data.decode('latin_1') )
> 
> 
> The unicode around this is superfluous.

Worse, it's an error.  utf-8 needs to go into a stream
of 8-bit bytes, not a Unicode string.

John Nagle
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: getting terminal display size?

[Jean-Paul Calderone]

On 25 Feb 2007 16:53:17 -0800, jeff <[EMAIL PROTECTED]> wrote:
>I looked around a lot on the internet and couldn't find out how to do
>this, how do I get the sizer (in rows and columns) of the view?
>

Assuming you're talking about something vaguely *NIXy, you want something
like what's being done here:

http://twistedmatrix.com/trac/browser/trunk/twisted/conch/scripts/conch.py#L361

Jean-Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: getting terminal display size?

[Grant Edwards]

On 2007-02-26, jeff <[EMAIL PROTECTED]> wrote:

> I looked around a lot on the internet and couldn't find out how to do
> this, how do I get the sizer (in rows and columns) of the view?

You use the TIOCGWINSZ ioctl() call on the tty device in question.

  import termios, fcntl, struct, sys

  s = struct.pack("", 0, 0, 0, 0)
  fd_stdout = sys.stdout.fileno()
  x = fcntl.ioctl(fd_stdout, termios.TIOCGWINSZ, s)
  print '(rows, cols, x pixels, y pixels) =',
  print struct.unpack("", x)



-- 
Grant Edwards
[EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: getting terminal display size?

[Grant Edwards]

On 2007-02-26, Grant Edwards <[EMAIL PROTECTED]> wrote:
> On 2007-02-26, jeff <[EMAIL PROTECTED]> wrote:
>
>> I looked around a lot on the internet and couldn't find out how to do
>> this, how do I get the sizer (in rows and columns) of the view?
>
> You use the TIOCGWINSZ ioctl() call on the tty device in question.
>
>   import termios, fcntl, struct, sys
>
>   s = struct.pack("", 0, 0, 0, 0)
>   fd_stdout = sys.stdout.fileno()
>   x = fcntl.ioctl(fd_stdout, termios.TIOCGWINSZ, s)
>   print '(rows, cols, x pixels, y pixels) =',
>   print struct.unpack("", x)

You may also want to handle the WINCH signal so that you know
when the window size has been changed.

-- 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: modifying a list while iterating through

[George Sakkis]

On Feb 25, 8:12 pm, [EMAIL PROTECTED] wrote:
> consider the following working loop where Packet is a subclass of
> list, with Packet.insert(index, iterable) inserting each item in
> iterable into Packet at consecutive indexes starting at index.
>
> i=0
> while(i if packet[i:i+5]==Packet("01110"):
> packet.insert(i, "0")
> i+=10 #skip the 5 bits inserted, and skip the 5 bits just
> checked bc overlap should not trigger insertion
> else: i+=1
>
> is there a way to do this more elegantly?  seems like a big kludge.

Unless I missed something, this is a simple string replacement:

''.join(packet).replace('01110', '011100')


George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: modifying a list while iterating through

[attn . steven . kuo]

On Feb 25, 5:12 pm, [EMAIL PROTECTED] wrote:
> consider the following working loop where Packet is a subclass of
> list, with Packet.insert(index, iterable) inserting each item in
> iterable into Packet at consecutive indexes starting at index.
>
> i=0
> while(i if packet[i:i+5]==Packet("01110"):
> packet.insert(i, "0")
> i+=10 #skip the 5 bits inserted, and skip the 5 bits just
> checked bc overlap should not trigger insertion
> else: i+=1
>
> is there a way to do this more elegantly?  seems like a big kludge.


If Packet consists of '0's and '1's, then it may be
easier to convert to, or base the class on str (strings):

packet = "101010011100111010001"
print "BEFORE ", packet
li = packet.split("01110")
packet = "011100".join(li)
print "AFTER  ", packet

--
Hope this helps,
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weakref problem: no way to get original object from proxy object?

[Gabriel Genellina]

En Sun, 25 Feb 2007 19:07:38 -0300, John Nagle <[EMAIL PROTECTED]>  
escribió:

> Is there some way to get a strong ref to the original object back
> from a weakref proxy object?  I can't find any Python function to do  
> this.
> ".ref()" doesn't work on proxy objects.

Add a method to the original class that just returns self.

About your backref function: Yes, I usually use something like that, it's  
annoying having to handle None always.

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RegExp performance?

[Kirk Sluder]

In article <[EMAIL PROTECTED]>,
 Christian Sonne <[EMAIL PROTECTED]> wrote:

> Thanks to all of you for your replies - they have been most helpful, and 
> my program is now running at a reasonable pace...
> 
> 
> I ended up using r"\b\d{9}[0-9X]\b" which seems to do the trick - if it 
> turns out to misbehave in further testing, I'll know where to turn :-P

Anything with variable-length wildcard matching (*+?) is going to 
drag your performance down. There was an earlier thread on this very 
topic.  Another stupid question is how are you planning on handling 
ISBNs formatted with hyphens for readability?

In general I've found the following factors to be critical in 
getting good performance from re:

1: Reducing the number of times you call re.match or re.search.
2: Reducing the number of bytes that re has to search through.
3: Minimizing the use of wildcards in the expression.

If you can pre-filter your input with string.find before running 
re.match you will improve performance quite a bit compared to 
running re expressions over all 10 pages.  I played around a bit and 
attached some example code below searching over 21K of text for the 
ISBN number.  testPrefilter() runs about 1/5th the execution time of 
line-by-line re calls or a single re call over a 21K string.  
Interestingly this ratio scales up to something as big as Mobey 
Dick.  

The searchLabels() functions below beats all the other functions by 
searching for "ISBN", or "International Book" and then using RE on 
the surrounding 500 bytes.  You might also try searching for 
"Copyright" or "Library of Congress" since most modern books will 
have it all on the same page.  

A caveat here is that this only works if you can find a reasonably 
unique string at or near what you want to find with re.  If you need 
to run re.search on every byte of the file anyway, this isn't going 
to help.  

---
timing test code
---

#!/usr/bin/env python

from timeit import Timer
import re

textString = """The text of a sample page using with an ISBN 10
number ISBN 0672328976 and some more text to compare."""

#add the full text of Mobey Dick to make the search functions
#work for their bread.
fileHandle= open("./mobey.txt")
junkText = fileHandle.readlines()
junkText.append(textString)
textString=''.join(junkText)
#print textString

#compile the regex
isbn10Re = re.compile(r"\b\d{9}[0-9X]\b")

def testPrefilter():
"""Work through a pre-loaded array running re only on lines 
containing ISBN""" 
for line in junkText:
#search for 'ISBN"
if (line.find('ISBN') > -1):
thisMatch = isbn10Re.search(line)
if thisMatch:
return thisMatch.group(0)
  
def testNofilter():
"""Run re.search on every line."""
for line in junkText:
#seaching using RE
thisMatch = isbn10Re.search(line)
if thisMatch:
return thisMatch.group(0)


def testFullre():
"""Run re.search on a long text string."""
thisMatch = isbn10Re.search(textString)
if thisMatch:
return thisMatch.group(0)

def searchLabels():
#identify some text that might be near an ISBN number.
isbnLabels=["ISBN",
   "International Book"]

#use the fast string.find method to locate those 
#labels in the text
isbnIndexes = [textString.find(x) for x in isbnLabels]

#exclude labels not found in the text.
isbnIndexes = [y for y in isbnIndexes if y > -1]

#run re.search on a 500 character string around the text label.
for x in isbnIndexes:
thisMatch=isbn10Re.search(textString[x-250:x+250])
return thisMatch.group(0)



#print searchLabels()
#print testPrefilter()
#print testNofilter()
t = Timer("testNofilter()","from __main__ import testNofilter")
print t.timeit(100)
u = Timer("testPrefilter()","from __main__ import testPrefilter")
print u.timeit(100)
v = Timer("testFullre()","from __main__ import testFullre")
print v.timeit(100)
w = Timer("searchLabels()", "from __main__ import searchLabels")
print w.timeit(100)
-- 
http://mail.python.org/mailman/listinfo/python-list

Jobs: Lisp and Python programmers wanted in the LA area

[Tech HR]

http://www.smartcharter.com/jobs.html

Smart Charter Inc. is a dynamic new startup company aiming to 
revolutionize the purchase and sale of private aviation services. If you 
are ready for a challenging ground-floor opportunity with significant 
upside potential you've come to the right place. We are not your typical 
dotcom. We have a razor-sharp focus on an existing multi-billion-dollar 
market. We are well funded, and have the connections and the technical 
expertise we need to revolutionize an industry.

We are looking for people who are highly motivated and passionate about 
their work, and able to produce high quality code within a fast paced 
development environment.

We are hiring for the following positions:

   €  Senior software engineer -- Ideal candidate would have significant 
development experience, possibly an advanced degree in computer science 
or related field, experience developing planning & scheduling or 
operations software using linear programming and heuristic search 
methods. Proficiency in multiple languages including (but not limited 
to) C++, Python and Common Lisp would also be desirable.

   €  Windows software engineer -- This person will be responsible for 
integrating elements of our products into multiple existing Windows 
applications. An ideal candidate would have experience in development 
for the PC platform, in multiple tool suites, including Visual 
Studio.net Useful skills include the ability to work within existing 
interfaces, protocols, and code structures and to work with extensively 
with database applications.

   €  Web developer, senior web developer, system administrator -- Our 
website is currently a LAMP appication with P=Python. We are looking for 
bright motivated people who know or are willing to learn Python and/or 
Linux, Apache and MySQL system administration skills. (And if you want 
to convince us that we should switch over to Postgres, we're willing to 
listen.)

We are more interested in smarts, passion, and a willingness to learn 
new things than specific credentials. If you are interested in joining 
us drop us a line at: tech-hr at smartcharter.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on Dict

[Hendrik van Rooyen]

 "James Stroud" <[EMAIL PROTECTED]> wrote:


> Clement wrote:
> > Can any body tell how Dict is implemented in python... plz tell what
> > datastructure that uses
> > 
> 
> I think it uses a dict.

"Groan!" - this answer is like Microsoft documentation - 
while technically correct, it is useless...  : - )

The key of a dict entry is subjected to a bit of magic called 
a "Hashing algorithm" that yields a "bin" in which the data
is kept.

So to get hold of a dict entry always takes a similar time.

And its the quickest form of random access based on
a non integer key that you can get.

hth - Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on object scope?

[Hendrik van Rooyen]

 "hg" <[EMAIL PROTECTED]> wrote:


> [EMAIL PROTECTED] wrote:
> 
> > Hello everybody,
> > 
> > I have a (hopefully) simple question about scoping in python. I have a
> > program written as a package, with two files of interest. The two
> > files are /p.py and /lib/q.py

Make a third file for all the system wide globals, say param.py:

global r
r = 42

> > 
> > My file p.py looks like this:
> > 
> > ---
> > 
> > from lib import q
> > 
> > def main():

from param import *

> >   global r
> >   r = q.object1()
> >   s = q.object2()
> > 
> > if __name__ == "__main__":
> >   main()
> > 
> > ---
> > 
> > My file q.py in the subdirectory lib looks like this:

from param import *

> > 
> > class object1:
> >   t = 3
> > 
> > class object2:
> >   print r.t
> > 
> > ---
> > 
> > Python gives me an error, saying it can't recognize global name r.
> > However I define r as global in the top-level main definition! Can
> > anyone suggest how I can get around this, if I want to define and bind
> > global names inside of main() which are valid in all sub-modules?
> > 
> > Thanks very much for your help!
> 
> Might be wrong, but globals can only be global to the module they're
> declared in.

Correct.

> 
> I suggest you find another way such as passing your object as a parameter

or make the third file and import it everywhere you need them...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[Hendrik van Rooyen]

"Arnaud Delobelle" <[EMAIL PROTECTED]> wrote:


> On Feb 25, 6:00 pm, "Paddy" <[EMAIL PROTECTED]> wrote:
> > I blogged on finding a new-to-me feature of Python, in that you are
> > allowed to nnest parameter definitions:
> >
> > >>> def x ((p0, p1), p2):
> >
> > ... return p0,p1,p2
> > ...>>> x(('Does', 'this'), 'work')
> >
> > ('Does', 'this', 'work')
> 
> Reminds me of LeLisp! It had a similar feature.  IIRC you could write
> for example (I think 'df' was LeLisp for 'defun'):
> (df mycar (a . b) a)
> or
> (df mylist L L)
> or
> (df mycaadr (a (b . c) . e) b)
> 
> I didn't know that this was possible in python and it does surprise
> me.  It feels at odd with the python philosophy.
> 

Not at all - it much nicer than you think, and there is no "nesting" 
involved - Penguins carry their eggs on their feet.

The original function definition describes a function that has a two element
tuple as a first parameter, and something else as a second one.
The first two names provide a means of accessing the elements of the
tuple, instead of using slicing.

look at this:

>>> def f((a,b),c):
 return a,b,c

>>> tup = ('hi','there')
>>> f(tup,'foo')
('hi', 'there', 'foo')
>>> lis = ['hi','there']
>>> f(lis,'foo')
('hi', 'there', 'foo')
>>> d,e,f = f(lis,42)
>>> print d,e,f
hi there 42
>>> e
'there'
>>> s = 'go'
>>> f(s,'back')
('g', 'o', 'back')
>>>  

Long live duck typing...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weakref problem: no way to get original object from proxy object?

[John Nagle]

Gabriel Genellina wrote:
> En Sun, 25 Feb 2007 19:07:38 -0300, John Nagle <[EMAIL PROTECTED]>  
> escribió:
> 
>> Is there some way to get a strong ref to the original object back
>> from a weakref proxy object?  I can't find any Python function to do  
>> this.
>> ".ref()" doesn't work on proxy objects.
> 
> 
> Add a method to the original class that just returns self.

Yes, that works.  Thanks.

John Nagle
-- 
http://mail.python.org/mailman/listinfo/python-list

help regarding python and jsp

[chandrapsg]

Hi,
i am working with jsp ..
i wanna help regarding how to import or how to call  python modules to
jsp

if a piece of code is availabe will be very helpful for me



chandra

-- 
http://mail.python.org/mailman/listinfo/python-list

help regarding python and jsp

[chandrapsg]

Hi,
i am working with jsp ..
i wanna help regarding how to import or how to call  python modules to
jsp

if a piece of code is availabe will be very helpful for me



chandra

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on Dict

[Steven D'Aprano]

On Mon, 26 Feb 2007 07:15:43 +0200, Hendrik van Rooyen wrote:

>  "James Stroud" <[EMAIL PROTECTED]> wrote:
> 
> 
>> Clement wrote:
>> > Can any body tell how Dict is implemented in python... plz tell what
>> > datastructure that uses
>> > 
>> 
>> I think it uses a dict.
> 
> "Groan!" - this answer is like Microsoft documentation - 
> while technically correct, it is useless...  : - )

Which I think was deliberate. The Original Poster asked a reasonable
question in a stupid way (Python is case-insensitive -- are we supposed
to guess what a Dict is? what language is 'plz'?), and I believe James
deliberately gave a stupid answer to highlight that.

class Dict(dict):
pass

is what I thought of when I saw the question. I guess James might have
thought the same way.

Unfortunately, more often than not, the sort of person who uses 'plz' in
an email probably isn't going to recognise the subtleties of James' reply.


-- 
Steven D'Aprano 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rational numbers

[Gabriel Genellina]

En Sat, 24 Feb 2007 01:42:00 -0300, Martin Manns <[EMAIL PROTECTED]> escribió:

> On Fri, 23 Feb 2007 20:20:12 -0300
> "Gabriel Genellina" <[EMAIL PROTECTED]> wrote:
>
>> mx.Number.Rational is horribly broken. They break this rule:
>> a==b => hash(a)==hash(b)
>> so they can'b be used as dictionary keys, by example.
>
> I would be interested, under which circumstances the rule is broken.
> In my (few) tests, hash(a)==hash(b) worked.

I can't reproduce this issue with the latest mx experimental package, it  
appears to be fixed now. Also, I'm currently using Python 2.5 and it was  
2.3 by that time.

-- 
Gabriel Genellina

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[bearophileHUGS]

Virgil Dupras:

> Without the call example, I would have
> had a hard time to try to figure out what these extra brackets are
> for. For this reason, I think that an explicit unpack is more
> readable, and thus better.

I can't agree.

Bye,
bearophile

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RegExp performance?

[John Machin]

On Feb 26, 2:01 pm, Kirk  Sluder <[EMAIL PROTECTED]> wrote:
> In article <[EMAIL PROTECTED]>,
>  Christian Sonne <[EMAIL PROTECTED]> wrote:
>
> > Thanks to all of you for your replies - they have been most helpful, and
> > my program is now running at a reasonable pace...
>
> > I ended up using r"\b\d{9}[0-9X]\b" which seems to do the trick - if it
> > turns out to misbehave in further testing, I'll know where to turn :-P
>
> Anything with variable-length wildcard matching (*+?) is going to
> drag your performance down. There was an earlier thread on this very
> topic.  Another stupid question is how are you planning on handling
> ISBNs formatted with hyphens for readability?

According to the OP's first message, 2nd paragraph:
"""
(it should be noted that I've removed all '-'s in the string, because
they have a tendency to be mixed into ISBN's)
"""

Given a low density of ISBNs in the text, it may well be better to
avoid the preliminary pass to rip out the '-'s, and instead:

1. use an RE like r"\b\d[-\d]{8,11}[\dX]\b" (allows up to 3 '-'s
inside the number)

2. post-process the matches: strip out any '-'s, check for remaining
length == 10.

Another thought for the OP: Consider (irrespective of how you arrive
at a candidate ISBN) validating the ISBN check-digit.

Cheers,
John

-- 
http://mail.python.org/mailman/listinfo/python-list