Abend with cls.__repr__ = cls.__str__ on Windows.

[J Peyret]

This gives a particularly nasty abend in Windows - "Python.exe has
stopped working", rather than a regular exception stack error.  I've
fixed it, after I figured out the cause, which took a while, but maybe
someone will benefit from this.

Python 2.6.5 on Windows 7.



class Foo(object):
pass

#def __str__(self):  #if you have this defined, no abend
#return "a Foo"

Foo.__repr__ = Foo.__str__   # this will cause an abend.

#Foo.__str__ = Foo.__repr__  #do this instead, no abend

foo = Foo()

print str(foo)



I suspect that object.__str__ is really object.__repr__ by default, as
they both print out the same string, so that this doesn't make any
sense.

What was I trying to achieve?  Leveraging __str__ to debug instances
in lists and containers.  In that case, __repr__ is called and I
usually do:

class Foo(object):
  def __str__(self):
 return "a foo"
  def __repr__(self):  #was trying to avoid this.
 return str(self)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Abend with cls.__repr__ = cls.__str__ on Windows.

[J Peyret]

On Mar 17, 9:37 pm, Terry Reedy  wrote:
> On 3/17/2011 10:00 PM, Terry Reedy wrote:
>
> > On 3/17/2011 8:24 PM, J Peyret wrote:
> >> This gives a particularly nasty abend in Windows - "Python.exe has
> >> stopped working", rather than a regular exception stack error. I've
> >> fixed it, after I figured out the cause, which took a while, but maybe
> >> someone will benefit from this.
>
> >> Python 2.6.5 on Windows 7.
>
> >> class Foo(object):
> >> pass
>
> >> Foo.__repr__ = Foo.__str__ # this will cause an abend.
>
> > 2.7.1 and 3.2.0 on winxp, no problem, interactive intepreter or IDLE
> > shell. Upgrade?
>
> To be clear, the above, with added indent, but with extra fluff (fixes)
> removed, is exactly what I ran. If you got error with anything else,
> please say so. Described behavior for legal code is a bug. However,
> unless a security issue, it would not be fixed for 2.6.
>
> --
> Terry Jan Reedy

Nope, that is it.  No need to upgrade, nor is there any urgency.  I
was just surprised to see it fail so brutally, is all.  I've only
encountered 2 or 3 core Python bugs at that level in about 13 yrs of
coding in it, so thought I'd post it, especially as it is so easy to
replicate.

Txs for your help

I actually have another really weird behavior in this program, but
haven't figured out yet what causes it so it is hard to tell if it's
an error on my part or another system bug.  I'll post it if I can
isolate a system error.

FWIW, what I am doing is using Configparser to assemble a bunch of
classes together to provide a reporting/diffing engine for database
comparisons.  The object compositions are all defined in an .ini file.

I've found Python + ini files are a great match for creating truly
flexible programs.  SQL queries, template strings and regular
expression patterns can be stored in the ini file and are easy to
modify without touching the core python code.  Pass a different ini
file => different behavior.  Xml is overkill for this and plays really
badly with relational <,> operators.

This is the next step for me, defining mostly separate classes and
assembling and initializing them based on ini file configuration info.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PostgreSQL vs MySQL (was Re: How to handle sockets - easily?)

[J Peyret]

On Mar 16, 10:19 am, a...@pythoncraft.com (Aahz) wrote:
> In article ,

> >always recommend people to use PostgreSQL, though; which is superior in
> >almost every way, especially the C client library and the wire protocol.)
>
> Can you point at a reference for the latter?  I have been trying to
> convince my company that PG is better than MySQL.
> --

Well, my $.02 worth is that, about 3 yrs ago, on 5.0x-5.1x, I was
truly appalled by the sheer level of stupidity displayed by MySQL's
handling of a moderately complex UPDATE SQL query with a correlated
subquery.

(Let's say this was a 7 out of 10 complexity, with your standard
selects being 1-3 max and a nightmare update query with all sorts of
correlated subqueries would be a 9.  I am first of all a database
programmer, so queries are my world).

Not only did MySQL mangle the query because it didn't understand what
I was asking, it thrashed the data during the update and committed
it.  And, when I reviewed the query once again, I found I had
mismatched parentheses, so it wasn't even syntaxically correct.  Truly
scary.

DB2 + SQLBase punted for years on correlated subqueries  Ex:  "update
ORDERS where x=y and exists (select 1 from ORDERS where )".  The DB2 engine doesn't know how to handle them, so it
tells you to get lost.

MySQL is not smart enough to recognize it's over its head and instead
makes a best effort.  To me it looks like a database that will get you
80% there and steadfastly refuse the last 20%, assuming you need
really clever queries.  PG was much cleaner in behavior, though a pain
to install, especially on Windows.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Abend with cls.__repr__ = cls.__str__ on Windows.

[J Peyret]

On Mar 18, 2:15 pm, Carl Banks  wrote:

> Multiple people reproduce a Python hang/crash yet it looks like no one
> bothered to submit a bug report
>
> I observed the same behavior (2.6 and 3.2 on Linux, hangs) and went
> ahead and submitted a bug report.
>
> Carl Banks

Speaking for myself, I've only put in one bug report on an OSS
project.  Won't do it again.

Firefox, some kinda "attribute works with html, but blows up on xhtml"
issue.  Nothing that was really xhtml related, but their Bugzilla
folks pulled out some unconvincing RFC crap claiming that it was out
of scope for xhtml as opposed to html so they wouldn't fix it.

Their choice.  4 yrs ago.

Still getting occasional emails from Bugzilla about that "we won't fix
it for you" bug.  At least 2 dozen over the years.  Nothing about
fixing it, just status churn.

If I ever specifically work on an OSS project's codeline, I'll post
bug reports, but frankly that FF example is a complete turn-off to
contributing by reporting bugs.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Abend with cls.__repr__ = cls.__str__ on Windows.

[J Peyret]

On Mar 18, 6:55 pm, Carl Banks  wrote:
> On Mar 18, 5:31 pm, J Peyret  wrote:
>
> > If I ever specifically work on an OSS project's codeline, I'll post
> > bug reports, but frankly that FF example is a complete turn-off to
> > contributing by reporting bugs.
>
> You probably shouldn't take it so personally if they don't agree with
> you.  But it's ok, it's not unreasonable to call attention to (actual)
> bugs here.
>

No, I didn't take the refusal personally at all, though it seemed
kinda glib to xhtml-not-hmtl out of a bug.  Such is life.

What I object to is getting auto-spammed for years after that because
I was silly enough to enter my email on their database and then being
notified about every single event on that bug.

> I was surprised, though, when several people confirmed but no one
> reported it, especially since it was a crash, which is quite a rare
> thing to find.  (You should feel proud.)
>
> Carl Banks

Yes, that's why I reported it.  Python is extremely stable so it
seemed worthwhile.  And I figured someone on the list would know its
bug quotient.

Impressed that there is already a bit of a proposed approach on the
entry to fix it, you guys are great.

But... I rather thought __repr__ was going to disappear in 3.x, with
just __str__ sticking around.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: multi-Singleton-like using __new__

[J Peyret]

I think the metaclass stuff is a bit too black magic for a pretty
simple requirement.

Txs in any case for showing me the __init__ issue, I wasn't aware of
it.

Here's a workaround - not exactly elegant in terms of OO, with the
isInitialized flag, but it works.

>>> class RDFObject(object):
... cache ={}
... isInitialized = False
... def __new__(cls,uri,*args,**kwds):
... try:
... return cls.cache[uri]
... except KeyError:
... print "cache miss"
... res = cls.cache[uri] = object.__new__(cls)
... return res
... def __init__(self,uri):
... if self.isInitialized:
... return
... print "__init__ for uri:%s" % (uri)
... self.isInitialized = True
...
>>> r1 = RDFObject(1)
cache miss
__init__ for uri:1
>>> r1b = RDFObject(1)
>>> r2 = RDFObject(2)
cache miss
__init__ for uri:2
>>> r2b = RDFObject(2)
>>> print r2b
<__main__.RDFObject object at 0x87a9f8c>
>>>print r2
<__main__.RDFObject object at 0x87a9f8c>

Some things to keep in mind:

- Might as well give uri its place as a positional param.  Testing
len(*args) is hackish, IMHO.

- Same with using try/except KeyError instead of in cls.cache.
Has_key might be better if you insist on look-before-you-leap, because
'in cls.cache' probably expends to uri in cls.cache.keys(), which can
be rather bad for perfs if the cache is very big.  i.e. dict lookups
are faster than scanning long lists.

- I took out the threading stuff - dunno threading and I was curious
if that was causing __init__ twice.  It wasn't, again txs for showing
me something I dinna know.

- isInitialized is as class variable.  __init__ looks it up from the
class on new instances, but immediately rebinds it to the instance
when assigning self.isInitialized = True.  On an instance that gets re-
__init__-ed, self.isInitialized exists, so the lookup doesn't
propagate up to the class variable (which is still False).

Cheers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: multi-Singleton-like using __new__

[J Peyret]

On Feb 8, 5:38 pm, Freek Dijkstra <[EMAIL PROTECTED]> wrote:

If you want to subclass, my initial example did not cover that.  This
will, or at least, I don't have any problems with similar code:

 ... def __new__(cls,uri,*args,**kwds):
... try:
... return cls.cache[(cls,uri)]  #notice that I added the
class itself as a key.
... except KeyError:
... print "cache miss"
... res = cls.cache[(cls,uri)] = object.__new__(cls)
... return res

Your mileage apparently varies, but I'm still not sold on using a
metaclass.  Just because it is a bit fancy for the requirements,
IMHO.  Later you may have cached/singleton classes that really need a
metaclass for something else, with subclasses of _those_ that don't
want caching.  That's definitely just my $.02 there, I don't
necessarily expect others to share my prejudices and misgivings.

Anyway, I should post answers more often - I learned more today than
by asking questions ;-)  Txs all, for the has_key/in performance
pointers, they are exactly contrary to what I would have thought.

Cheers
-- 
http://mail.python.org/mailman/listinfo/python-list

Coverage.py reporting and UML tools - what exists already?

[J Peyret]

I got coverage.py to work after somewhat of a difficult start...

Hint:  if moving your code from Windows to Linux and if running
'coverage.py -r mymodule.py' causes SyntaxError/SyntaxException, the
'flip' utility is your friend to deal with removing those nasty \r\n
newlines that are preventing coverage.py from working.

... and I can generate annotated files.  Great, but it would be really
nice to have an quick overview of untested code.

One Java tool I've used in the past is Cobertura, which can output its
coverage reports in html format.

http://cobertura.sourceforge.net/sample/

I was wondering if there is anything similar to dress up coverage.py
annotation files?  Wouldn't seem to be very difficult to html-ize the
files a bit.  I can probably take a, feeble, stab at it, but I'd
rather not reinvent any wheels.


Second question:

I'd like a basic UML tool to draw up some interaction diagrams
(Collaboration/Sequence) on some of my hairier pieces of code.  I
think of it more as documentation/brainstorming diagrams than anything
else.  I.e. something that helps me remember how things work and can
help me spot refactoring opportunities.

Things I don't care about:

- document most of my code - this is for the truly complex 5-10% of
interactions
- generating diagrams from code or code from diagrams
- static class diagrams
- descriptions doing the whole UML hog - type declarations,
stereotypes, etc...

What I do care about:

- sketching basic diagrams manually as quickly as possible

Most of the software I've seen takes great pride in reverse
engineering or generating code, often of the Java variety.  In fact,
everything looks dauntingly complex/powerful.  Anybody seen the
equivalent of an UML/CRC-card aware blackboard?  Something as
trivially dumb/easy as the early Visio/ABC Flowcharter?

I've looked at ArgoUML, BoaConstructor and UMLet in the past and
didn't really like them.  What about Dia?  Looking at UML from a
Python / post-coding documentation angle, what seems to fit the bill
best?

I am on Linux or Windows, using PyDev on Eclipse.

Cheers
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Looking for a Python Program/Tool That Will Add Line Numbers to a txt File

[J Peyret]

On Feb 14, 8:50 am, "W. Watson" <[EMAIL PROTECTED]> wrote:

(snip)

> I thought this might be more difficult judging by a long ago experience with 
> Java.

(snip)

+1 QOTW
-- 
http://mail.python.org/mailman/listinfo/python-list

usage of .encode('utf-8','xmlcharrefreplace')?

[J Peyret]

Well, as usual I am confused by unicode encoding errors.

I have a string with problematic characters in it which I'd like to
put into a postgresql table.
That results in a postgresql error so I am trying to fix things with
.encode

>>> s = 'he Company\xef\xbf\xbds ticker'
>>> print s
he Company�s ticker
>>>

Trying for an encode:

>>> print s.encode('utf-8')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position
10: ordinal not in range(128)

OK, that's pretty much as expected, I know this is not valid utf-8.
But I should be able to fix this with the errors parameter of the
encode method.

>>> error_replace = 'xmlcharrefreplace'

>>> print s.encode('utf-8',error_replace)
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position
10: ordinal not in range(128)

Same exact error I got without the errors parameter.

Did I mistype the error handler name?  Nope.

>>> codecs.lookup_error(error_replace)


Same results with 'ignore' as an error handler.

>>> print s.encode('utf-8','ignore')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position
10: ordinal not in range(128)

And with a bogus error handler:

print s.encode('utf-8','bogus')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position
10: ordinal not in range(128)

This all looks unusually complicated for Python.
Am I missing something incredibly obvious?
How does one use the errors parameter on strings' encode method?

Also, why are the exceptions above complaining about the 'ascii' codec
if I am asking for 'utf-8' conversion?

Version and environment below.  Should I try to update my python from
somewhere?

./$ python
Python 2.5.1 (r251:54863, Oct  5 2007, 13:36:32)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2

Cheers
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: usage of .encode('utf-8','xmlcharrefreplace')?

[J Peyret]

OK, txs a lot.  I will have to think a bit more about you said, what I
am doing and how encode/decode fits in.

You are right, I am confused about unicode.  Guilty as charged.

I've seen the decode+encode chaining invoked in some of the examples,
but not the rationale for it.
Also doesn't help that I am not sure what encoding is used in the data
file that I'm using.

I didn't set out to "hide" the original error, just wanted to simplify
my error posting, after having researched enough to see that
encode/decode was part of the solution.
Adding the db aspect to the equation doesn't really help much and I
should have left it out entirely.

FWIW:

 
invalid byte sequence for encoding "UTF8": 0x92
HINT:  This error can also happen if the byte sequence does not match
the encoding expected by the server, which is controlled by
"client_encoding".

column is a varchar(2000) and the "guilty characters" are those used
in my posting.

Txs again.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: usage of .encode('utf-8','xmlcharrefreplace')?

[J Peyret]

On Feb 18, 10:54 pm, 7stud <[EMAIL PROTECTED]> wrote:

> One last point: you can't display a unicode string.  The very act of
> trying to print a unicode string causes it to be converted to a
> regular string.  If you try to display a unicode string without
> explicitly encode()'ing it first, i.e. converting it to a regular
> string using a specified secret code--a so called 'codec', python will
> implicitly attempt to convert the unicode string to a regular string
> using the default codec, which is usually set to ascii.

Yes, the string above was obtained by printing, which got it into
ASCII format, as you picked up.
Something else to watch out for when posting unicode issues.

The solution I ended up with was

1) Find out the encoding in the data file.

In Ubuntu's gedit editor, menu 'Save As...' displays the encoding at
the bottom of the save prompt dialog.

ISO-8859-15 in my case.

2) Look up encoding corresponding to ISO-8859-15 at

http://docs.python.org/lib/standard-encodings.html

3) Applying the decode/encode recipe suggested previously, for which I
do understand the reason now.

#converting rawdescr
#from ISO-8859-15 (from the file)
#to UTF-8 (what postgresql wants)
#no error handler required.
decodeddescr = rawdescr.decode('iso8859_15').encode('utf-8')

postgresql insert is done using decodeddescr variable.

Postgresql is happy, I'm happy.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Deviation from object-relational mapping (pySQLFace)

[J Peyret]

On Oct 12, 8:19 am, [EMAIL PROTECTED] wrote:

> I would like to get some opinions on this approach.
> Thanks.

I realize I will be minority here, but...

I've never quite understood why folks want to repeat the database's
metadata in XML files.  I've gotten much better results just using
plain ol' SQL throughout, sprinkled in with generated-on-the-fly SQL.


1.  A select clause identifies what is coming back from the db in the
cursor's description.  20 lines of code shoves that in a dictionary
for each row for any result set.  'Select * from ' works 90% of
the time for 1 table queries.  What does XML add?

2.  Inserts and deletes are relatively trivial to derive from
INFORMATION SCHEMA lookups on any given table and templates can be
generated for them.  Updates are admittedly less trivial, but not
horribly so.

3.  Query parameters can be added by simple %()s embedded in
the query templates.That works great with dictionaries.  You can
extract them with a regular expression and replace them with '?' and a
list, if your DB-API flavor requires that.

4.  Plain ol' SQL can be cut and pasted in a query editor and can be
tested there.

5.  If you unit test somewhat aggressively, any db-schema changes will
result in unhappy queries dying because they don't see the columns
that they expect in the resultsets.  That keeps your Python code in
synch without feeding a layer of XML cruft.

6.  XML is plain nasty for "simple local usage" where you don't need
to communicate with a 3rd party app or module.  Conversely, XML is
great when you need to communicate data "somewhere else, potentially
with recursive and nested structures".

7.  ANSI SQL is actually quite portable, if you know what to avoid
doing.

8.  Last, but not least.  Performance.

In complex processing on a database with large volumes, the last thing
you want to do is to fetch data to your client codeline, process it
there, and spew it back to the database.  Instead you want to shoot
off series of updates/deletes/insert-selects queries to the server and
you want to rely on set-based processing rather than row-by-row
approaches.  How do ORMs+XML help here?

My biggest hassle has been managing connection strings and catching
the weird Exception structures every Python db module figures it has
to re-implement, not the SQL itself.

Granted, if this were Java, you would need special data transfer
objects to encapsulate the results.  But is not Java.  And, also
granted, I _enjoy_ coding in SQL rather than trying to hide from it,
so YMMV.

Bottom line:  SQL is extremely dynamic in nature, even more so than
Python.  Why shackle it to static XML files?

P.S.

SQL Alchemy _is_ something I've been meaning to look at, because it
seems like they also _like_ SQL.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie Question: How to use a .pth file on a Macintosh

[J Peyret]

Hmmm, for lack of a better response, here are some suggestions, based
on what I've seen on Windows+Linux.

#1 put the .pth in the site-packages directory (this is what I do on
Linux).  I think Python considers it special and looks for pth.

you can probably get that directory from doing

import sys
for i in sys.path:
print i

#2 put the .pth in the directory where the python executable is
located (this is less messy on Windows than on Linux or OS X)

which python (run on the OS X terminal command shell) should give you
an idea of where that is.

#3 Another idea is to use the Finder to find _other_ *.pth files that
may exist and put yours in the same location.

Best of luck.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python is slow?

[J Peyret]

On Sep 23, 8:31 am, [EMAIL PROTECTED] wrote:

Guys, this looks like a great data structure/algo for something I am
working on.

But... where do I find some definitions of the original BK-tree idea?
I looked through Amazon
and only a few books mention something like BK-Tree and these are
mostly conference minutes books, at ungodly prices.

I also did a quick Google on it and there isn't that much about the
subject.

http://blog.notdot.net/archives/30-Damn-Cool-Algorithms,-Part-1-BK-Trees.html

is the one I mostly saw referred.

So... 2 questions:

1.  More bk-tree references?  I can follow the code, but some
understanding of the background would be nice.

2.  What, if any, is a good book to understand the basic of fuzzy/
string matching?  Proximity/affinity problems?  Or, more generally, a
good book on advanced algorithms?

No, I don't wanna read Knuth's just yet, something more modern/easy to
follow maybe?  Something like 'Programming Collective Intelligence',
ISBN 0596529325, would be very nice, though it is perhaps a bit too
specific in its applications.  Books using Java or C are fine.  Lisp,
hmmm, well... I have trouble reading its notation, sorry.

Cheers

JLuc
--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP Proposal

[J Peyret]

On Sep 25, 12:24 pm, [EMAIL PROTECTED] wrote:

> def whoisthethief("List" x):
>   return iknowit(x)
>
> def whoisthethief("String" x, "String" y):
>   return iknowit([x,y])
>

I dunno if this is very Pythonic in nature, but I've done things like
rebinding methods dynamically.

ex:

>>> def test(a):
... return a * 2
...
>>> print test(2)
4
>>> def test2(b):
... return b * 3
...
>>> test = test2
>>> print test(2)
6

In your case

def myNewThiefIdentifier(x,y):
   return "Dunno"

whoisthethief = myNewThiefIdentifier


Which method would this affect (i.e. which 'whoisthethief')?

I assume you could figure it out, given a match on the signature, but
how much work would this require from the developer and the
interpreter?

(apologies in case Google Groups decide to multipost)
--
http://mail.python.org/mailman/listinfo/python-list

Re: twenty years ago Guido created Python

[J Peyret]

On Dec 31 2009, 2:06 pm, Steve Howell  wrote:
> FYI:
>
> http://twitter.com/gvanrossum
>
> Python is a truly awesome programming language.  Not only is Guido a
> genius language designer, but he is also a great project leader.  What
> an accomplishment.  Congratulations to everybody who has contributed
> to Python in the last two decades!

Notwithstanding all of the above, which are all true, having met
Guido, I say he is a genuinely nice human being.  Go BSD derivatives.


-- 
http://mail.python.org/mailman/listinfo/python-list

slightly OT - newbie Objective-C resources for experienced Python users

[J Peyret]

I'm starting to look at the iPhone SDK and I'd like to know of
resources on the Net that approach that language with a Pythonic
mindset.  Mind you, I want to code Objective-C, not pine about Python
not being on the iPhone either.  The kind of elegant simple code that
a good Python coder who also knows C well would code in C.

For example, valueForKey seems to map well to dynamic coding a la
__getattr__, but a Java programmer would be all over the getter/
setters, design patterns and frameworks instead.  I want to know about
simplicity, introspection and dynamic stuff, not Java-translated-to-
Objective-C.

Any recommendations?
-- 
http://mail.python.org/mailman/listinfo/python-list