Re: Functional style programming in python: what will you talk about if you have an hour on this topic?

[J Kenneth King]

Anthony Kong  writes:

> (My post did not appear in the mailing list, so this is my second try. 
> Apology if it ends up posted twice)
>
> Hi, all,
>
> If you have read my previous posts to the group, you probably have some idea 
> why I asked this question.
>
> I am giving a few presentations on python to my colleagues who are mainly 
> java developers and starting to pick up python at work.
>
> 
> So I have picked this topic for one of my presentation. It is because 
> functional programming technique is one of my favorite in my bag  of python 
> trick. It also takes me to the rabbit hole of the functional programming 
> world, which is vastly more interesting than the conventional procedural/OO 
> languages.
> 
>
> I think I will go through the following items:
>
> itertools module
> functools module
> concept of currying ('partial')
>
>
> I would therefore want to ask your input e.g.
>
> Is there any good example to illustrate the concept? 
> What is the most important features you think I should cover?
> What will happen if you overdo it?
>
>
> Cheers

You might also want to cover gotchas like Python's references.

If you have the time I'd introduce weakref too.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Refactor/Rewrite Perl code in Python

[J Kenneth King]

Steven D'Aprano  writes:

> On Sun, Jul 24, 2011 at 7:29 PM, Shashwat Anand 
> wrote:
>
>> How do I start ?
>> The idea is to rewrite module by module.
>> But how to make sure code doesn't break ?
>
> By testing it.
>
> Read up on "test driven development".
>
> At this point, you have this:
>
> Perl modules: A, B, C, D
> Python modules: none
> Python tests: none
>
> Now, before you rewrite each Perl module in Python, first write a good,
> comprehension test suite in Python for that module. You need to have tests
> for each function and method. Test that they do the right thing for both
> good data and bad data. If you have functional requirements for the Perl
> modules, use that as your reference, otherwise use the Perl code as the
> reference.
>
> For example, this might be a basic test suite for the len() built-in
> function:
>
>
> for empty in ([], {}, (), set([]), ""):
> if len(empty) != 0:
> raise AssertionError('empty object gives non-zero len')
>
> for n in range(1, 5):
> if len("x"*n) != n:
> raise AssertionError('failure for string')
> for kind in (list, tuple, set):
> obj = kind([None]*n)
> if len(obj) != n:
> raise AssertionError('failure for %s' % obj)
>
> if len({'a': 1, 'b': None, 42: 'spam'}) != 3:
> raise AssertionError('failure for dict')
>
> for bad_obj in (23, None, object(), 165.0, True):
> try:
> len(bad_obj)
> except TypeError:
> # Test passes!
> pass
> else:
> # No exception means len() fails!
> raise AssertionError('failed test')
>
>
>
> Multiply that by *every* function and method in the module, and you have a
> moderately good test suite for module A.
>
> (You may want to learn about the unittest module, which will help.)
>
> Now you have:
>
> Perl modules: A, B, C, D
> Python modules: none
> Python tests: test_A
>
> Now re-write the Python module, and test it against the test suite. If it
> fails, fix the failures. Repeat until it passes, and you have:
>
> Perl modules: A, B, C, D
> Python modules: A
> Python tests: test_A
>
> Now you can be confident that Python A does everything that Perl A does.
> Possibly *better* than the Perl module, since if you don't have a test
> suite for it, it probably has many hidden bugs.
>
> Continue in this way with the rest of the modules. At the end, you will have
> a full test suite against the entire collection of modules.

A very sane approach.

I currently support a large legacy application written in C++ by
extending it with Python.  We managed to do this by wrapping the legacy
application with a Python interface using the Boost::Python libraries.
It has allowed us to embrace and extend the legacy code and replace bits
of it one piece at a time while keeping the whole application running.

Now I am not aware of any Python bindings to the Perl interpreter, but
if there is some FFI library that makes it possible this approach might
be viable for your project.

The trade-off of this approach is the extra complexity of maintaining
another interface in your code.  The beneift is that you can avoid
re-writing a large amount of code up front.  This works particularly
well when the legacy system has a rather large user base and you don't
have a robust test suite for the legacy code.

One book I found particularly useful: 

http://www.amazon.ca/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052

hth
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Two questions about style and some simple math

[J Kenneth King]

Spoofy  writes:

> ..  ..
>
> 2.
>
> For maintaining the character attributes I creates a seperate class. I
> wonder weather this is an "overuse" of OO (instead of just making the
> attributes plain variables of the Char class) and if the way I wrote
> this is OK (somehow this looks cool to me but maybe too "showy"?)
>
> class Attributes(object):
> ATTRIBUTES = {"attack": 0, "defence": 0, "ability": 0, "courage":
> 0, "condition": 0}
> def __init__(self, **kwargs):
> self.__dict__.update(self.ATTRIBUTES)
> for arg in kwargs:
> if arg not in self.ATTRIBUTES:
> raise ValueError("Unkown character attribute '%s'" %
> arg)
> self.__dict__[arg] = kwargs[arg]
>
>
> Again, I appreciate any tips. I you need more code (for the bigger
> picture or such), just ask.
>
> Thanks in advance

I think the first part has been covered well.

If you want an opinion, this class isn't "showy" at all, rather it is
ugly and unnecessary.

Firstly it's bad because:

1. ATTRIBUTES is still modifiable after insantiation. This breaks the
restriction you're expressing in __init__

2. You want to avoid modifying __dict__ directly if you can. It bypasses
the whole point of using named attributes.

What you'd really want in a case where a class has a restricted set of
attributes is __slots__. Classes defined with it have no __dict__ and
thus attributes cannot be dynamically added to them after
instanciation (a __slots__ defined class will raise an exception in this
case).

However, even in this case, it doesn't make sense to encapsulate the
attributes of your game's Character objects in this way. Your "Hero"
class should have its attributes directly associated to it: Hero.health,
Hero.defence, and so forth. If you want to encapsulate a common set of
attributes available to a "class" of objects, you'd create a "Character"
class with those general attributes, sub-class your NPC's and Hero from
it, and specialize those sub-classes as needed.

HTH,

j_king
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pyro deadlock

[J Kenneth King]

MatthewS  writes:

> I'd like to know if the following behavior is expected and can be
> avoided: I have a Pyro server object that maintains a queue of work,
> and multiple Pyro worker objects that take work off the queue by
> calling a method on the server (get_work) and then return the work to
> the server by calling another method (return_result).
>
> The problem is that I experience a deadlock condition in the
> return_result method.
>
> Is this deadlock expected or should Pyro be able to synchronize the
> remote calls to server's callback method when called from multiple
> workers at the same time? If so any hints are appreciated.
>
> /Matthew

Depends on how you're implementing the work/result queues.

The Queue module is a good start.
--
http://mail.python.org/mailman/listinfo/python-list

Re: I'm a python addict !

[J Kenneth King]

Linuxguy123  writes:

> I just started using python last week and I'm addicted.
>
> I hate Perl.  I never did learn to use it with any competence.  I has to
> be the most obfuscated, cryptic language I've ever seen.  Making it
> "object oriented" only makes it worse !
> ..  ..

I program full-time in Python, so I share your excitement and
enthusiasm. But bashing Perl like that doesn't make you sound very
smart. I'm probably one of the very few Python programmers who came from
(and still occassionally) use Perl. I've written non-trivial programs in
it and from my experience I can tell you that it's actually a great
language. The Moose object system is well beyond Python's class
system. But I guess you wouldn't know that.

So yay for Python, but don't get in the habit of criticising that which
you do not know.
--
http://mail.python.org/mailman/listinfo/python-list

Re: I'm a python addict !

[J Kenneth King]

J Kenneth King  writes:

> Linuxguy123  writes:
>
>> I just started using python last week and I'm addicted.
>>
>> I hate Perl.  I never did learn to use it with any competence.  I has to
>> be the most obfuscated, cryptic language I've ever seen.  Making it
>> "object oriented" only makes it worse !
>> ..  ..
>
> I program full-time in Python, so I share your excitement and
> enthusiasm. But bashing Perl like that doesn't make you sound very
> smart. I'm probably one of the very few Python programmers who came from
> (and still occassionally) use Perl. I've written non-trivial programs in
> it and from my experience I can tell you that it's actually a great
> language. The Moose object system is well beyond Python's class
> system. But I guess you wouldn't know that.
>
> So yay for Python, but don't get in the habit of criticising that which
> you do not know.

I just realized this might become flame-bait. Please disregard the
flamey bits. :(
--
http://mail.python.org/mailman/listinfo/python-list

Re: Recommendation for a small web framework like Perl's CGI::Application to run as CGI?

[J Kenneth King]

excord80  writes:

> I need to make a small, relatively low-traffic site that users can
> create accounts on and log into. Scripts must run as cgi (no
> mod_python or FastCGI is available). Can anyone recommend a small and
> simple web framework for Python, maybe similar to Perl's
> CGI::Application?
>
> Or would it just be better to roll my own?

web.py

pretty simple. lots of deployment options.

ends up building into your own framework as your application grows.
--
http://mail.python.org/mailman/listinfo/python-list

Re: What is wrong in my list comprehension?

[J Kenneth King]

Chris Rebert  writes:

> Python 2.6 (r26:66714, Nov 18 2008, 21:48:52)
> [GCC 4.0.1 (Apple Inc. build 5484)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
 bool(-1)
> True
>
> str.find() returns -1 on failure (i.e. if the substring is not in the
> given string).
> -1 is considered boolean true by Python.

That's an odd little quirk... never noticed that before.

I just use regular expressions myself.

Wouldn't this be something worth cleaning up? It's a little confusing
for failure to evaluate to boolean true even if the relationship isn't
direct.
--
http://mail.python.org/mailman/listinfo/python-list

Re: What is wrong in my list comprehension?

[J Kenneth King]

Chris Rebert  writes:

> Python 2.6 (r26:66714, Nov 18 2008, 21:48:52)
> [GCC 4.0.1 (Apple Inc. build 5484)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
 bool(-1)
> True
>
> str.find() returns -1 on failure (i.e. if the substring is not in the
> given string).
> -1 is considered boolean true by Python.

That's an odd little quirk... never noticed that before.

I just use regular expressions myself.

Wouldn't this be something worth cleaning up? It's a little confusing
for failure to evaluate to boolean true even if the relationship isn't
direct.
--
http://mail.python.org/mailman/listinfo/python-list

Re: What is wrong in my list comprehension?

[J Kenneth King]

Stephen Hansen  writes:

>>> str.find() returns -1 on failure (i.e. if the substring is not in the
>>> given string).
>>> -1 is considered boolean true by Python.
>>
>> That's an odd little quirk... never noticed that before.
>>
>> I just use regular expressions myself.
>>
>> Wouldn't this be something worth cleaning up? It's a little confusing
>> for failure to evaluate to boolean true even if the relationship isn't
>> direct.
>
> But what would you clean it up to?
>
> str.find can return 0 ... which is a *true* result as that means it
> finds what you're looking for at position 0... but which evaluates to
> boolean False. The fact that it can also return -1 which is the
> *false* result which evaluates to boolean True is just another side of
> that coin.
>
> What's the options to clean it up? It can return None when it doesn't
> match and you can then test str.find("a") is None... but while that
> kinda works it also goes a bit against the point of having boolean
> truth/falsehood not representing success/failure of the function. 0
> (boolean false) still is a success.
>
> Raising an exception would be a bad idea in many cases, too. You can
> use str.index if that's what you want.
>
> So there's not really a great solution to "cleaning it up" . I
> remember there was some talk in py-dev of removing str.find entirely
> because there was no really c, but I have absolutely no idea if they
> ended up doing it or not.
>
> --S

(Sorry all for the multiple post... my gnus fudged a bit there)

That's the funny thing about integers having boolean contexts I
guess. Here's a case where 0 actually isn't "False." Any returned value
should be considered "True" and "None" should evaluate to "False." Then
the method can be used in both contexts of logic and procedure.

(I guess that's how I'd solve it, but I can see that implementing it
is highly improbable)

I'm only curious if it's worth cleaning up because the OP's case is one
where there is more than one way to do it.

However, that's not the way the world is and I suppose smarter people
have discussed this before. If there's a link to the discussion, I'd
like to read it. It's pedantic but fascinating no less.
--
http://mail.python.org/mailman/listinfo/python-list

Re: What is wrong in my list comprehension?

[J Kenneth King]

Chris Rebert  writes:

> Python 2.6 (r26:66714, Nov 18 2008, 21:48:52)
> [GCC 4.0.1 (Apple Inc. build 5484)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
 bool(-1)
> True
>
> str.find() returns -1 on failure (i.e. if the substring is not in the
> given string).
> -1 is considered boolean true by Python.

That's an odd little quirk... never noticed that before.

I just use regular expressions myself.

Wouldn't this be something worth cleaning up? It's a little confusing
for failure to evaluate to boolean true even if the relationship isn't
direct.
--
http://mail.python.org/mailman/listinfo/python-list

Re: What is wrong in my list comprehension?

[J Kenneth King]

Chris Rebert  writes:

> Python 2.6 (r26:66714, Nov 18 2008, 21:48:52)
> [GCC 4.0.1 (Apple Inc. build 5484)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
 bool(-1)
> True
>
> str.find() returns -1 on failure (i.e. if the substring is not in the
> given string).
> -1 is considered boolean true by Python.

That's an odd little quirk... never noticed that before.

I just use regular expressions myself.

Wouldn't this be something worth cleaning up? It's a little confusing
for failure to evaluate to boolean true even if the relationship isn't
direct.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Added-value of frameworks?

[J Kenneth King]

Matimus  writes:

> On Feb 4, 8:08 am, Gilles Ganault  wrote:
>> Hello
>>
>> If I wanted to build some social web site such as Facebook, what do
>> frameworks like Django or TurboGears provide over writing a site from
>> scratch using Python?
>>
>> Thank you for your feedback.
>
> Why not just look at the frameworks themselves and see what they have
> to offer. Django and Turbogears both have pretty good tutorials. You
> can be up and running in 20 minutes with each one. You be the judge.
>
> The frameworks provide a lot of boilerplate code that I would rather
> not write. They are probably more secure and scalable than something I
> would come up with. You also get many extras for free. I think in both
> of the frameworks you mention you get an administrative back end for
> free. Other people have created apps/plugins that you can use with
> those frameworks. So, for example, you might be able to borrow the
> code to help you add a forum to your site.
>
> I'm not sure I know the advantage of not using a framework. Unless "I
> get to write more code" is an advantage. Creating your own framework
> might be fun, but if you really just want a website don't do more work
> than you need to.
>
> Matt

In other words, it boils down to what you get paid to do.

If you're being paid for a frob (the product, in this case a website)
then you use a frob-maker (a framework).

If you're being paid to make frobs, then you make the frob-maker.

Most frob-makers are good at producing frobs of a certain kind. Just
choose the frob-maker that makes the frobs you need.

In rare circumstances you'll need a really customized frob. Call on me
when you get there. ;)
--
http://mail.python.org/mailman/listinfo/python-list

Re: [Web 2.0] Added-value of frameworks?

[J Kenneth King]

Bruno Desthuilliers  writes:

> Gilles Ganault a écrit :
>> Hello
>>
>> If I wanted to build some social web site such as Facebook, what do
>> frameworks like Django or TurboGears provide over writing a site from
>> scratch using Python?
>
> Quite a lot of abstractions and factorisation of the boilerplate code,
> a known way to organize your application, and possibly a good
> integration of the usual components (and wrt/ Django, a customizable
> yet fairly usable OOTB admin interface). For simple to mediumly
> complex applications, this can mean more than 80% of the grunt
> work. The counterpart is mostly learning and understanding the
> framework, which means you may not save that much time on a first
> project - but it can really pay off then. One of my coworker started
> this morning a (really simple...) project which acceptance tests are
> to begin on monday, and we are all quite confident he'll deliver on
> time, thanks to Django.

Well the big negative is when you application design starts expanding
past the framework design. A percolator makes a decent cup of coffee but
it really isn't meant for making fancy lattes. This is where the 90/10
rule will catch you if you're not careful.

"90% of the effort will be spent on the last 10% of the work."
--
http://mail.python.org/mailman/listinfo/python-list

Re: Flattening lists

[J Kenneth King]

mk  writes:

> Hello everybody,
>
> Any better solution than this?
>
> def flatten(x):
> res = []
> for el in x:
> if isinstance(el,list):
> res.extend(flatten(el))
> else:
> res.append(el)
> return res
>
> a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]]
> print flatten(a)
>
>
> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>
> Regards,
> mk

http://mail.python.org/pipermail/python-list/2005-July/330367.html
--
http://mail.python.org/mailman/listinfo/python-list

[ANN] TracShell 0.1 released

[J Kenneth King]

I tend to work a lot with Trac for project management and have always
found the browser interface to be a productivity killer. I always
wanted a simple command-line interface to Trac, but having never found
one I found a little free time and got off my laurels to make one.

TracShell 0.1 is an early release, but it works. So far you can only
query and view tickets, but planned updates include the obvious
ability to create and edit tickets. Future plans will allow browsing
of comments, change histories, attachments, and so forth.

Please consider it really beta. The code needs a little tidying up
around the edges. If you find any bugs, please report them and I'll
fix them ASAP. Ideas, suggestions, and contributions are welcome.

http://code.google.com/p/tracshell/

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: TracShell 0.1 released

[J Kenneth King]

Krzysztof Retel  writes:

> On 12 Feb, 14:06, J Kenneth King  wrote:
>> I tend to work a lot with Trac for project management and have always
>> found the browser interface to be a productivity killer. I always
>> wanted a simple command-line interface to Trac, but having never found
>> one I found a little free time and got off my laurels to make one.
>>
>> TracShell 0.1 is an early release, but it works. So far you can only
>> query and view tickets, but planned updates include the obvious
>> ability to create and edit tickets. Future plans will allow browsing
>> of comments, change histories, attachments, and so forth.
>>
>> Please consider it really beta. The code needs a little tidying up
>> around the edges. If you find any bugs, please report them and I'll
>> fix them ASAP. Ideas, suggestions, and contributions are welcome.
>>
>> http://code.google.com/p/tracshell/
>>
>> Cheers.
>
> Nice work.
>
> Maybe it would be worth building at some point vim or emacs interface.
>
> Krzysztof

Thanks. :)

I considered a native editor interface, but since both vi and emacs
can run a shell within a buffer, I decided to go with a more flexible
solution.

However, if someone wanted to, the actual code fetching and displaying
the data isn't tied directly to the shell interface. It could be
possible to dispatch calls from a native editor front-end if it
doesn't mind spawning a python interpreter (or keeping one running)
for each request.

Anyway, thanks for checking it out. Hope you find it useful. I do. :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: is there a project running (GUI Builder for Python ) ?

[J Kenneth King]

azrael  writes:

> To be honest, in compare to Visual Studio, Gui Builders for wx
> widgets are really bad.

That's because Visual Studio is a Microsoft product to build
interfaces for Microsoft products.

wx on the other hand is cross platform and ergo, much more
complicated.

> Do you know if there is any project curently running for GUI
> builders, maybe for WX, That I could maybe join.

You could help wx. Make your own gui builder if you think you could do
better. There's also GTK and Qt... and probably many others.
--
http://mail.python.org/mailman/listinfo/python-list

Re: [ANN] TracShell 0.1 released

[J Kenneth King]

J Kenneth King  writes:

> I tend to work a lot with Trac for project management and have always
> found the browser interface to be a productivity killer. I always
> wanted a simple command-line interface to Trac, but having never found
> one I found a little free time and got off my laurels to make one.
>
> TracShell 0.1 is an early release, but it works. So far you can only
> query and view tickets, but planned updates include the obvious
> ability to create and edit tickets. Future plans will allow browsing
> of comments, change histories, attachments, and so forth.
>
> Please consider it really beta. The code needs a little tidying up
> around the edges. If you find any bugs, please report them and I'll
> fix them ASAP. Ideas, suggestions, and contributions are welcome.
>
> http://code.google.com/p/tracshell/
>
> Cheers.

Just added the ability to create tickets, more features forthcoming.

I highly recommend anyone using this tool to stick to the latest svn
versions. I'll package the feature-complete stables going forward.

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: is there a project running (GUI Builder for Python ) ?

[J Kenneth King]

gc_ott...@yahoo.ca writes:

> ..I come from Delphi, and compared to Delphi, even Visual Studio
>> vanishes ;-)
> ...I don't even notice the difference between Delphi (which
> I'm still using)
>> and wxPython.
>>
>> I think this story happened to other people to,
>> so instead of putting a lot of effort in designing and maintaining a GUI
>> builders,
>> it might be better to choose another solution.
>>
>> btw, the idea I used, can be seen 
>> here]http://mientki.ruhosting.nl/data_www/pylab_works/pw_gui_support.html
>> and the code can be found 
>> herehttp://code.google.com/p/pylab-works/downloads/list
>>
>> cheers,
>> Stef
>
> You know, 10 or more years ago both Borland and Microsoft got it right
> when they incorporated a GUI with an IDE in their Delphi and Visual
> Basic products. As wonderful as the Python language is, it is very
> much a work in progress when compared to the ease of use of the
> aforementioned products. These products revolutionized the industry
> with their "Rapid Applications Development" (RAD).
>
> Python reminds me of a toolbox filled with a multitude of single use
> tools when all you need is a Skilsaw and a combination screwdriver.
> Gord

... So use the combination screwdriver.

Python isn't all things to all people. It is what it is.
--
http://mail.python.org/mailman/listinfo/python-list

Musings: Using decorators to reduce duplicate exception handling

[J Kenneth King]

I recently started a project called TracShell
(http://code.google.com/p/tracshell) where I make heavy use of the
xmlrpclib core module.

When the number of RPC calls was small, wrapping each call in try/except
was acceptable. However, this obviously will duplicate code all over the
place. There are only ever two exceptions that the module itself will
throw: xmlrpclib.ProtocolError and xmlrpclib.Fault -- both very useful,
but not show stoppers.

To combat the duplication, my clever idea was to use a function
decorator to wrap any function that used xmlrpclib calls:

def catch_errors(fn):
"""
A decorator to catch typical xmlrpclib exceptions
"""
def wrapped(*args, **kwargs):
try:
return fn(*args, **kwargs)
except xmlrpclib.ProtocolError, e:
print "There was a problem communicating with the server."
print "URL: %s" % e.url
print "Headers: %s" % e.headers
print "Error code: %d" % e.errcode
print "Error message: %s" % e.errmsg
print "Please file a report with the TracShell developers."
pass
except xmlrpclib.Fault, e:
print "A fault ocurred"
print "Fault code: %d" % e.faultCode
print "Fault string: %s" % e.faultString
print "If you think this message is the result of an error,"
print "please file a report with the TracShell developers."
pass
return wrapped

Maybe I could rename the decorator to something meaningful, but besides
that it works pretty well. Now any function I write in my class that
uses RPC calls can be wrapped by this decorator and have the exception
handling done in a uniform way for these particular exceptions.

I was just curious if other Pythonistas out there used the same idea or
have a better one. Thoughts?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Musings: Using decorators to reduce duplicate exception handling

[J Kenneth King]

"Gabriel Genellina"  writes:

> En Tue, 17 Feb 2009 21:12:57 -0200, J Kenneth King
>   escribió:
>
>> I recently started a project called TracShell
>> (http://code.google.com/p/tracshell) where I make heavy use of the
>> xmlrpclib core module.
>>
>> When the number of RPC calls was small, wrapping each call in try/except
>> was acceptable. However, this obviously will duplicate code all over the
>> place. There are only ever two exceptions that the module itself will
>> throw: xmlrpclib.ProtocolError and xmlrpclib.Fault -- both very useful,
>> but not show stoppers.
>>
>> To combat the duplication, my clever idea was to use a function
>> decorator to wrap any function that used xmlrpclib calls:
>>
>> def catch_errors(fn):
>> """
>> A decorator to catch typical xmlrpclib exceptions
>> """
>> def wrapped(*args, **kwargs):
>> try:
>> return fn(*args, **kwargs)
>> except xmlrpclib.ProtocolError, e:
>> print "There was a problem communicating with the server."
>> print "URL: %s" % e.url
>> print "Headers: %s" % e.headers
>> print "Error code: %d" % e.errcode
>> print "Error message: %s" % e.errmsg
>> print "Please file a report with the TracShell developers."
>> pass
>> except xmlrpclib.Fault, e:
>> print "A fault ocurred"
>> print "Fault code: %d" % e.faultCode
>> print "Fault string: %s" % e.faultString
>> print "If you think this message is the result of an error,"
>> print "please file a report with the TracShell developers."
>> pass
>> return wrapped
>
> I don't like the idea of "hiding" an exception. The caller code
> doesn't  know an exception occurred, and just continue doing its work,
> with bogus  results... is this what you want? Also, you don't include
> the stack trace  - and usually it contains very valuable
> information. When your users start  "filing a report with the
> TracShell developers" and you feel clueless, a  stack trace is
> important (you don't have to show it on screen - a log file  is even
> better).
>
> If all you want is to customize the error message, use sys.except_hook
>
> Looking into the code, those "pass" statement are useless; and error
> messages are usually written to stderr instead of stdout.

Thanks for the ideas.

I haven't actually used this pattern in any projects before, I was just
looking for a way to reduce the number of common try/except statements I
needed to have in my code; especially when they all looked exactly the
same and were catching every single xml-rpc call littered throughout my
class.

sys.except_hook looks more like what I was aiming for. I like Lisp's
"restarts," where I can define error condition handlers in one place
without having to litter my code with error handling statements.

Cheers.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Musings: Using decorators to reduce duplicate exception handling

[J Kenneth King]

Cameron Simpson  writes:

> On 17Feb2009 15:12, J Kenneth King  wrote:
> | I recently started a project called TracShell
> | (http://code.google.com/p/tracshell) where I make heavy use of the
> | xmlrpclib core module.
> | 
> | When the number of RPC calls was small, wrapping each call in try/except
> | was acceptable. [...]
> | To combat the duplication, my clever idea was to use a function
> | decorator to wrap any function that used xmlrpclib calls:
> | def catch_errors(fn):
> [...]
> | Maybe I could rename the decorator to something meaningful, but besides
> | that it works pretty well. Now any function I write in my class that
> | uses RPC calls can be wrapped by this decorator and have the exception
> | handling done in a uniform way for these particular exceptions.
>
> My Python fu is still weak, but I had a similar issue last year and
> wrote a context manager, used thus:
>
>   with NoExceptions(handler):
>   ... do stuff that may break ...
>
> The constructor takes a handleException argument (which may be None, but it
> has to be explicit) to tune which exceptions are caught; default is
> everything. This was for a daemon thread that did RPC calls, so it was
> actually fairly important to never die; normally you'd want to catch only
> specific exceptions.
>
> Code:
>
>   class NoExceptions(object):
>   ''' A context manager to catch _all_ exceptions and log them.
>   Arguably this should be a bare try...except but that's syntacticly
>   noisy and separates the catch from the top.
>   '''
>
>   def __init__(self, handleException):
>   ''' Initialise the NoExceptions context manager.
>   The handleException is a callable which
>   expects (exc_type, exc_value, traceback)
>   and returns True or False for the __exit__
>   method of the manager.
>   If handleException is None, the __exit__ method
>   always returns True, suppressing any exception.
>   '''
>   self.__handler = handleException
>
>   def __enter__(self):
>   pass
>
>   def __exit__(self, exc_type, exc_value, traceback):
>   if self.__handler is not None:
>   return self.__handler(exc_type, exc_value, traceback)
>   if exc_type is not None:
>   print >>sys.stderr, "ignore %s" % (exc_type,)
>   return True
>
> It doesn't actually solve your duplication issue and is a bit of a niche
> application, but it's a lot shorter than an inline try/except and to my eye
> reads a little better, and it keeps the exception handling up front at the
> calling end where it is visible.
>
> Cheers,

Ah, that's pretty interesting. I've found a couple other decorator-style
recipes in the cookbooks online (http://code.activestate.com/recipes/408937/).

I think you see my point: every single RPC call needs to be caught just
in case because they are so volatile -- yet you're always looking for
the exact same exception everywhere... too noisy to have inline
try/except. I wonder if there are other use cases where Python
developers have found the need to abstract away repetitive exception
handling.

I think this is the first I've seen the newer Python context managers
being used (though it sneakily looks a lot like what Lisp calls,
"unwind-protect"). Neat idea.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python equivalent of Common Lisp Macros?

[J Kenneth King]

dpapathanasiou <[EMAIL PROTECTED]> writes:

> I'm using the feedparser library to extract data from rss feed items.
>
> After I wrote this function, which returns a list of item titles, I
> noticed that most item attributes would be retrieved the same way,
> i.e., the function would look exactly the same, except for the single
> data.append line inside the for loop.
>
> In CL, I could simply write a macro, then replace the data.append line
> depending on which attribute I wanted.
>
> Is there anything similar in Python?
>
> Here's the function:
>
> def item_titles (feed_url):
> """Return a list of the item titles found in this feed url"""
> data = []
> feed = feedparser.parse(feed_url)
> if feed:
> if len(feed.version) > 0:
> for e in feed.entries:
> data.append(e.title.encode('utf-8'))
> return data

No macros -- need the whole "code as data" paradigm which doesn't exist
in python.

The pythonic way to create the sort of abstraction you're looking for
would be to use the *attr functions and either a map of attribute
identities or inspect the attributes from the objects.

Some UNTESTED code...

def extract_entry(e, attrs=['title', 'datetime', 'article']):
out = []
for attr in attrs:
assert hasattr(e, attr)
out.append(getattr(e, attr))
return out

Using the inspect module would give an even more generic function, but
with more complex code of course.

Then your code would look more like:

def items(feed_url):
feed = feedparser.feed(feed_url)
if feed and len(feed) > 0:
return [extract_entry(entry) for entry in feed.entries]

Of course, I'm making liberal assumptions about what you're looking for
in your ouput here... but it's just an example, not a full
solution. YMMV. Hope it helps. :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: RELEASED Python 3.0 final

[J Kenneth King]

Barry Warsaw <[EMAIL PROTECTED]> writes:

> On behalf of the Python development team and the Python community, I  
> am happy to announce the release of Python 3.0 final.

Yay!

Thanks for all the great work.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.0 automatic decoding of UTF16

[J Kenneth King]

Johannes Bauer <[EMAIL PROTECTED]> writes:

> Traceback (most recent call last):
>   File "./modify.py", line 12, in 
> a = AddressBook("2008_11_05_Handy_Backup.txt")
>   File "./modify.py", line 7, in __init__
> line = f.readline()
>   File "/usr/local/lib/python3.0/io.py", line 1807, in readline
> while self._read_chunk():
>   File "/usr/local/lib/python3.0/io.py", line 1556, in _read_chunk
> self._set_decoded_chars(self._decoder.decode(input_chunk, eof))
>   File "/usr/local/lib/python3.0/io.py", line 1293, in decode
> output = self.decoder.decode(input, final=final)
>   File "/usr/local/lib/python3.0/codecs.py", line 300, in decode
> (result, consumed) = self._buffer_decode(data, self.errors, final)
>   File "/usr/local/lib/python3.0/encodings/utf_16.py", line 69, in
> _buffer_decode
> return self.decoder(input, self.errors, final)
> UnicodeDecodeError: 'utf16' codec can't decode bytes in position 74-75:
> illegal encoding

It probably means what it says: that the input file contains characters
it cannot read using the specified encoding.

Are you generating the file from python using a file object with the
same encoding? If not, then you might want to look at your input data
and find a way to deal with the exception.
--
http://mail.python.org/mailman/listinfo/python-list

Re: pydb 1.24

[J Kenneth King]

[EMAIL PROTECTED] (R. Bernstein) writes:

> This release is to clear out some old issues. It contains some
> bugfixes, document corrections, and enhancements. Tests were
> revised for Python 2.6 and Python without readline installed. A bug
> involving invoking from ipython was fixed. The "frame" command is a
> little more like gdb's. Exceptions are now caught in runcall().
>
> This is the last release contemplated before a major rewrite.
>
> download:
> https://sourceforge.net/project/showfiles.php?group_id=61395&package_id=175827
>
> bug reports:
> https://sourceforge.net/tracker/?group_id=61395&atid=497159

I watched the demo video, look forward to working with it. Any links to
that emacs front-end being used in the video?

Cheers and thanks!
--
http://mail.python.org/mailman/listinfo/python-list

Re: Are Django/Turbogears too specific?

[J Kenneth King]

Gilles Ganault  writes:

> Hi
>
> I'd like to rewrite a Web 2.0 PHP application in Python with AJAX, and
> it seems like Django and Turbogears are the frameworks that have the
> most momentum.
>
> I'd like to use this opportunity to lower the load on servers, as the
> PHP application wasn't built to fit the number of users hammering the
> servers now.
>
> I'm concerned, though, that these frameworks they may be too specific
> to the tasks they were originally developped for (news articles,
> AFAIK). Do you think I should just use eg. CherryPy and some basic
> AJAX?
>
> Thank you for any feedback.

They're not "specific" in the sense that they only build certain types
of applications. However, they do package their own batteries and
expect applications to be designed a certain way. As long as you drink
the kool-aid, everything is smooth sailing. That's what any
"framework" banks on -- being useful to 85% of the web applications;
not all. Even if they advertise themselves as such, it's just not
true.

My suggestion is web.py -- It too has its own set of conventions and
philosophies, but in my experience it is the most loosely coupled in
terms of internal dependencies. Import exactly what you want to use
and deploy it the way that suits you. It gives you batteries but you
have to pick and choose how to put the pieces together. Therefore, it
takes a little longer to get running, but IMO that is the most
flexibility you can ask for without writing your own servers and
frameworks.

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: If your were going to program a game...

[J Kenneth King]

Tokyo Dan  writes:

> If your were going to program a game in python what technologies would
> you use?
>
> The game is a board game with some piece animations, but no movement
> animation...think of a chess king exploding. The game runs in a
> browser in a window of a social site built around the game. The social
> site has login, chat, player stats, list of active games, etc. AND
> there is also be a desktop client that accesses the game server via
> the same communication mechanism (like an AIR-based desktop client/
> app) as the browser-based version - I guess using JSON/RPC.

Ever see chess.com?

I don't know what they're using in the backend, but the client is
entirely javascript.

You could probably roll your own python javascript compiler to suit your
needs. It could probably even build up your own DSL for writing these
games.

It's a worthwhile project and I think there might be support for it from
other developers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a better algorithm?

[J Kenneth King]

Kottiyath  writes:

> I have the following list of tuples:
> L = [(1, 2), (3, 4, 5), (6, 7)]
>
> I want to loop through the list and extract the values.
> The only algorithm I could think of is:
 for i in l:
> ...  u = None
> ...  try:
> ...   (k, v) = i
> ...  except ValueError:
> ...   (k, u, v) = i
> ...  print k, u, v
> -
> 1 None 2
> 3 4 5
> 6 None 7
> -
> But, this algorithm doesnt look very beautiful - like say -> for k, u,
> v in L:
> Can anyone suggest a better algorithm to get the values?

Just a note: this isn't really an algorithm problem. ;)

It's more of a grammar obstruction.

To make your code more simple, it would be nice if the assignment
operator would return, "None," in the case where there are too few
values to unpack from the right-operand of the assignment operator.

Aside from the typical workarounds that first came to mind, I started
wondering whether it was possible to expose the problem and solve it
directly.

Sadly, it appears to be non-trivial (or at least, well hidden from the
unwashed masses).

I'd be really curious if the unpacking machinery were exposed to the
developer. I started poking around the operator and types modules, but
the implementation isn't obvious. What methods are being called on the
operands by the assignment operator in the following statement:

a, b, c = some_tuple

I'm sorry if this line of inquiry is not very pythonic; but one is
curious if this is some python magic happening here. After using the
idiom for years I hadn't really thought about it much until recently.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a better algorithm?

[J Kenneth King]

Paul Rubin  writes:

> Kottiyath  writes:
>> I have the following list of tuples:
>> L = [(1, 2), (3, 4, 5), (6, 7)]
>> I want to loop through the list and extract the values.
>
> Others have suggested messy ways to code what you're asking.  At another
> level, that list format seems like a code smell.  You may be better off
> organizing the program so that
>
>L = [(1, None, 2), (3, 4, 5), (6, None, 7)]
>
> after which unpacking becomes trivial.

Very true. +1
--
http://mail.python.org/mailman/listinfo/python-list

Re: Noob question: Is all this typecasting normal?

[J Kenneth King]

"Gabriel Genellina"  writes:

> En Mon, 05 Jan 2009 02:03:26 -0200, Roy Smith  escribió:
>
>
>> The other day, I came upon this gem.  It's a bit of perl embedded in a
>> Makefile; this makes it even more gnarly because all the $'s get
>> doubled  to
>> hide them from make:
>>
>> define absmondir
>> $(shell perl -e ' \                                                
>>      sub absmon { my $$a = $$_[0]; \                                
>>             if ( $$^O =~ m/cygwin|MSWin32/i ) {                    
>>                      $$prefix = `/bin/mount -p|awk "NR==2{print
>> \\\$$1}"`;
>> chomp($$prefix); \
>>       $$a = ($$_[1]||"$(PWD)") . "/$$a" \
>>          unless ( $$a =~ m !^(:?$$prefix|/|[A-Za-z]:)! ); \
>>    } else { $$a = ($$_[1]||"$(PWD)") . "/$$a" unless ( $$a =~ m !^/!
>> );  } \
>>    return unslash(undot(undotdot($$a))); }; \
>> sub unslash ($$) { $$_[0] =~ s://+:/:g; $$_[0] =~ s:/$$::;
>> return($$_[0]);
>> }; \
>> sub undot ($$) { $$_[0]=~s:/\./:/:g; return ($$_[0]); }; \
>> sub undotdot ($$) { my $$in = $$_[0]; \
>>       return ( $$in =~ s:/[^/.][^/]*/\.\.::g )?undotdot($$in):$$in; }; \
>> print absmon("$(1)","$(2)"); \
>> ' )                                                                
>>      endef
>>
>> Barf-o-rama.  I know what it's supposed to do, and I still can't
>> figure  it out.
>
> Ouch! Me too, when I come to some piece of Perl code I've written some
> years ago, I invariably think "what's all this noise?". Never happens
> with  other languages I've used in the past.

I still occassion upon the chance to write some Perl, and even as a
full-time developer who works with Python for a living, I relish every
opportunity.

The funny thing is that I've never had the problem of writing code like
this in Perl. The example is a very poor use-case and doesn't reflect on
the useful/useless-ness of the language itself but more on the choices
of the implementor.

Perl is a very useful language overall and when used properly, very
powerful.
--
http://mail.python.org/mailman/listinfo/python-list

Re: image recogniton?

[J Kenneth King]

Li Han  writes:

> Hi! I know little about the computer image processing, and now I have
> a fancy problem which is how to read the time from the picture of a
> clock by programming ?  Is there anyone who can give me some
> suggestions?
> Thank!
>  Li Han

I do work in object recognition, and I would classify this as a rather
difficult problem. Not impossible of course, but you'll need some OCR to
read the clock face and some sort of magnitude vector feature to tell
which hand is which and the "general direction" is is pointing in.

Also, depends if we're talking digital clocks or analog. :)

The other problem is one of accuracy: depending on your input image,
even slight variances can change your results.

I'm curious as to what application the solution to this problem is
practical for all of its difficulty?
--
http://mail.python.org/mailman/listinfo/python-list

Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

[J Kenneth King]

Jonathan Gardner  writes:

> On Jan 6, 8:18 am, sturlamolden  wrote:
>> On Jan 6, 4:32 pm, mark  wrote:
>>
>> > I want to implement a internal DSL in Python. I would like the syntax
>> > as human readable as possible.
>>
>> Also beware that Python is not Lisp. You cannot define new syntax (yes
>> I've seen the goto joke).
>
> This isn't really true. You can, for instance, write a program (in
> Python) that takes your pseudo-Python and converts it into Python.
> This is what a number of templating libraries such as Mako do.

Which is not even close to being the same.

Lisp - the program source is also the data format

Python - the program source is a string

I could go on a really long rant about how the two are worlds apart, but
I'll let Google tell you if you're really interested.
--
http://mail.python.org/mailman/listinfo/python-list

Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

[J Kenneth King]

Jonathan Gardner  writes:

> On Jan 6, 12:24 pm, J Kenneth King  wrote:
>> Jonathan Gardner  writes:
>> > On Jan 6, 8:18 am, sturlamolden  wrote:
>> >> On Jan 6, 4:32 pm, mark  wrote:
>>
>> >> > I want to implement a internal DSL in Python. I would like the syntax
>> >> > as human readable as possible.
>>
>> >> Also beware that Python is not Lisp. You cannot define new syntax (yes
>> >> I've seen the goto joke).
>>
>> > This isn't really true. You can, for instance, write a program (in
>> > Python) that takes your pseudo-Python and converts it into Python.
>> > This is what a number of templating libraries such as Mako do.
>>
>> Which is not even close to being the same.
>>
>> Lisp - the program source is also the data format
>>
>> Python - the program source is a string
>>
>> I could go on a really long rant about how the two are worlds apart, but
>> I'll let Google tell you if you're really interested.
>
> I get that Lisp is special because you can hack on the reader as it is
> reading the file in. This is strongly discouraged behavior, as far as
> I know, despite the number of cute hacks you can accomplish with it.

It is generally discouraged unless there's a reason for it.

> But consider that this really isn't different than having a program
> read in the lisp-with-modification source and spitting out pure lisp,
> to be read by an honest-to-gosh lisp program later.
>
> If that's the case, then Lisp and Python really aren't that different
> in this regard, except that you don't have the option of modifying the
> reader as it reads in the file.

I think you are missing the distinction.

Lisp expressions are also data structures. A Lisp expression can be
passed to functions and macros to be operated on before being
executed. When you're writing Lisp source, you're basically looking at
the AST on one level and when you start writing macros for your program,
you're creating a "DSL" or interface to that AST. Lisp source is
eventually expanded to a giant list that is consed by the evaluator (as
far as I understand it. I'm just getting into the compiler stuff
myself).

Consider:

(my-func 1 2 3)

This is just a list, the "primitive" data-type in Lisp! This piece of
"data" can be operated on by other bits of Lisp code because it is just
a list as far as Lisp is concerned.

In contrast, Python source is a string that needs to be parsed into
bytecode which is then run through the interpreter. The AST is
completely hidden from the source author. Python expressions are not
data types either and hence no macros -- I can't write a python function
that generates python code at compile time. I can only write a python
program that parses some other string and generates code that can be run
by another interpreter.

Consider:

for i in range(0, 100):
do_something_interesting(i)

That's a pretty straight forward Python expression, but I can't do
anything with it -- it's not a unit of data, it's a string.

The distinction is not subtle by any means.
--
http://mail.python.org/mailman/listinfo/python-list

Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

[J Kenneth King]

Kay Schluehr  writes:

> On 7 Jan., 16:50, J Kenneth King  wrote:
>
>> Python expressions are not
>> data types either and hence no macros -- I can't write a python function
>> that generates python code at compile time.
>
> Have you ever considered there are languages providing macros other
> than Lisp?

Of course.

> Macros have nothing to do with homoiconcity.

Not directly, no.

>> I can only write a python
>> program that parses some other string and generates code that can be run
>> by another interpreter.
>
> No, it is the same interpreter and it is also possible to modify
> python parsers on the fly. This is just not possible with Pythons
> builtin parser.

PyPy is probably the best bet when/if it gets finished.

>
>
>>
>> Consider:
>>
>> for i in range(0, 100):
>>     do_something_interesting(i)
>>
>> That's a pretty straight forward Python expression, but I can't do
>> anything with it -- it's not a unit of data, it's a string.
>>
>> The distinction is not subtle by any means.
--
http://mail.python.org/mailman/listinfo/python-list

Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

[J Kenneth King]

Jonathan Gardner  writes:

> On Jan 7, 9:16 am, "Chris Mellon"  wrote:
>>
>> The OP wants a Ruby-style DSL by which he means "something that lets
>> me write words instead of expressions". The ruby syntax is amenable to
>> this, python (and lisp, for that matter) syntax is not and you can't
>> implement that style of internal DSL in those languages.
>>
>> The answer to the OP is "you can't - use Ruby or modify your requirements".
>>
>
> As far as putting the code into Python, yeah, you can't put it in
> Python. The best you can do is store it in a string and then interpret
> the string with some function later on.

That's what I'm saying.

It seems we're defining "DSL" in two different ways.

You can't write a DSL in Python because you can't change the syntax and
you don't have macros.

You can write a compiler in Python that will compile your "DSL."

As another poster mentioned, eventually PyPy will be done and then
you'll get more of an "in-Python" DSL.
--
http://mail.python.org/mailman/listinfo/python-list

Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

[J Kenneth King]

Kay Schluehr  writes:

> On 8 Jan., 16:25, J Kenneth King  wrote:
>
>> As another poster mentioned, eventually PyPy will be done and then
>> you'll get more of an "in-Python" DSL.
>
> May I ask why you consider it as important that the interpreter is
> written in Python?

I don't think it's important for Python to have a meta-circular
interpreter (though it can't hurt).

> I see no connection between PyPy and syntactical
> Python extensions and the latter isn't an objective of PyPy. You can
> write Python extensions with virtually any Python aware parser.
> M.A.Lemburg already mentioned PLY and PLY is used for Cython. Then
> there is ANTLR which provides a Python grammar. I also know about two
> other Python aware parsers. One of them was written by myself.

Because... there is no connection to see? I never mentioned any such
relation.

DSL's tend to be a natural side-effect of languages which can manipulate
their own expressions without extensive parsing.

Creating a new parser that can generate Python AST's is certainly a
valid approach (and probably the easiest one). It's not the only one.

It depends on your definition of a DSL.

My definition isn't satisfied with creating a parser, and so my answers
reflect that.
--
http://mail.python.org/mailman/listinfo/python-list

Re: PIL problem

[J Kenneth King]

bfrederi <[EMAIL PROTECTED]> writes:

> I am having a problem using PIL. I am trying to crop and image to a
> square, starting from the center of the image, but when I try to crop
> the image, it won't crop. Here are the relevant code snippets:
>
> ### Function I am testing ###
> def create_square_image(file_name):
> """ Creates a thumbnail sized image and turns it into a square """
> image = Image.open(open(file_name))
>
> size_tuple = image.size
> width = size_tuple[0]
> height = size_tuple[1]
>
> square_length = 75
>
> x1 = (width / 2) - (square_length / 2)
> x2 = x1 + square_length
> y1 = (height / 2) - (square_length / 2)
> y2 = y1 + square_length
>
> image.crop((x1,y1,x2,y2))
> image.save(file_name, "JPEG")

def create_square_image(filename, size=75):
file = open(filename, 'rb')
image = Image.open(file)
w, h = image.size

x1 = (w / 2) - (size / 2)
x2 = x1 + size
y1 = (h / 2) - (size / 2)
y2 = y1 + size

thumb = image.crop((x1,y1,x2,y2))
thumb.save("thumb_" + filename, "JPEG")

...

of course PIL has a thumbnail method that does this type of stuff.
--
http://mail.python.org/mailman/listinfo/python-list

Re: PYTHON WORKING WITH PERL ??

[J Kenneth King]

Pat <[EMAIL PROTECTED]> writes:

> Sean DiZazzo wrote:
>> On Sep 29, 12:44 pm, "Blubaugh, David A." <[EMAIL PROTECTED]>
>> wrote:
>>> Sir,
>>>
>>> You are absolutely correct.  I was praying to G_d I did not have to
>>> slaughter my project's source code in this manner.  However, like life
>>> itself, I was given legacy source code (i.e. someone else errors to fix)
>>> in Perl.  However, I have just found out that there is a way to import
>>> the Perl interpreter within Python!!!  I now believe I can utilize
>>> python as the main platform to develop the project upon !!  
>>>
>>> Thanks,
>>>
>>> David
>>>
>>> -Original Message-
>>> From: D'Arcy J.M. Cain [mailto:[EMAIL PROTECTED]
>>> Sent: Monday, September 29, 2008 1:32 PM
>>> To: Blubaugh, David A.
>>>
>>> Cc: [EMAIL PROTECTED]
>>> Subject: Re: PYTHON WORKING WITH PERL ??
>>>
>>> On Mon, 29 Sep 2008 13:16:14 -0400
>>> "Blubaugh, David A." <[EMAIL PROTECTED]> wrote:
 I was wondering if it was possible to have a situation where a
 programming project would utilized BOTH python and perl?  Such as
 utilizing python for internet programming and then utilize perl for
 text processing and systems programming?  Is this even feasible???
>>> I don't see why not but I also question if it is a good idea.  Once you
>>> have all your objects and low level methods written in Python it just
>>> makes sense to re-use them rather than trying to duplicate the
>>> functionality in another language.
>>>
>>> Of course, sometimes we don't have control over our entire environment
>>> so yes, you can mix them if you have to.
>>>
>>
>>
>> Rewrite everything in python.  Save yourself now...while you still
>> can.
>>
>> ~Sean
>
> Trust me. Sean is absolutely correct. I'm currently in the process of
> converting a large Perl project to Python (and learning Python at the
> same time) and the improvement in code is incredible.  After you learn
> Python, you'll come to despise Perl.

Depends on the person -- I still love Perl, but program in Python
every day at work.

Python is great, but don't be mistaken: it's not the one language to
rule them all. No language is (except maybe Lisp).

But yay for converting a project to python. :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: PYTHON WORKING WITH PERL ??

[J Kenneth King]

Joshua Kugler <[EMAIL PROTECTED]> writes:

> Pat wrote:
>>> Rewrite everything in python.  Save yourself now...while you still
>>> can.
>>> 
>>> ~Sean
>> 
>> Trust me. Sean is absolutely correct. I'm currently in the process of
>> converting a large Perl project to Python (and learning Python at the
>> same time) and the improvement in code is incredible.  After you learn
>> Python, you'll come to despise Perl.
>
> I'm not a Python fan-boy, but I'm going to have to agree with Sean too.  I
> had been using Perl on-and-off for some 10 years, when I finally had to
> write a 500-ish line script in Perl for some data processing.  It got messy
> in a hurry.  Is this a hash? Is it a reference to a hash? Is it a reference
> to a hash of array references?  Gaaah!  I rewrote it in Python, and it was
> so much easier to keep all my data structures straight.
>
> j

Perl just gives you a lot of rope.

You can do a lot of neat things with rope. ;)
--
http://mail.python.org/mailman/listinfo/python-list

python extensions: including project local headers

[J Kenneth King]

Hey everyone,

I'm working on a python extension wrapper around Rob Hess'
implementation of a SIFT feature detector. I'm working on a
computer-vision based project that requires interfacing with Python at
the higher layers, so I figured the best way to handle this would be in
C (since my initial implementation in python was ungodly and slow). 

I can get distutils to compile the extension and install it in the
python path, but when I go to import it I get the wonderful exception:

ImportError: /usr/lib/python2.5/site-packages/pysift.so: undefined
symbol: _sift_features

Of course, _sift_features is a function defined in his header that I'm
#including in my extension.

His sources are sitting in my project root under sift/ while my source
is under src/ -- My setup.py is as follows:

[code]

from distutils.core import setup, Extension

pysift = Extension('pysift',
   include_dirs = ['sift/include'],
   sources = ['src/pysift.c'],
   extra_link_args = ['-lm', '-lcv', '-lcxcore',
  '-lhighgui', '-lcvaux'])

setup(name = 'pysift',
  version = '0.0',
  description = 'A SIFT feature detection package',
  author = 'James Kenneth King',
  author_email = "[EMAIL PROTECTED]",
  url = "http://agentultra.com/";,
  long_description = """
  A python extension package for detecting SIFT
  features using Rob Hess' C implementation.

  http://web.engr.oregonstate.edu/~hess/

  Original SIFT feature descriptors by David Lowe
  and patented by the University of British Columbia.
  """,
  ext_modules = [pysift])

[/code]

And the include to Rob's header file is on the second line of pysift.c:

#include "sift.h"

The weird thing (to me in my somewhat hackish knowledge of C) is that I
can use all the #defines from sift.h with no complaints from the
preprocessor (in fact, there are no complaints at all from the compiler
when compiling the extension module).

Once I get this bugger working, I'll be setting up a project page to
share sources and will also be releasing extension wrappers to the
OpenCV libraries.

I've never released any code before so any help getting this right and
proper for the community would be greatly appreciated.

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: python extensions: including project local headers

[J Kenneth King]

Robert Kern <[EMAIL PROTECTED]> writes:

> Philip Semanchuk wrote:
>>
>> On Oct 23, 2008, at 11:36 AM, J Kenneth King wrote:
>>
>>>
>>> Hey everyone,
>>>
>>> I'm working on a python extension wrapper around Rob Hess'
>>> implementation of a SIFT feature detector. I'm working on a
>>> computer-vision based project that requires interfacing with Python at
>>> the higher layers, so I figured the best way to handle this would be in
>>> C (since my initial implementation in python was ungodly and slow).
>>>
>>> I can get distutils to compile the extension and install it in the
>>> python path, but when I go to import it I get the wonderful exception:
>>>
>>> ImportError: /usr/lib/python2.5/site-packages/pysift.so: undefined
>>> symbol: _sift_features
>>
>>
>> Kenneth,
>> You're close but not interpreting the error quite correctly. This
>> isn't an error from the compiler or preprocessor, it's a library
>> error. Assuming this is dynamically linked, your OS is reporting
>> that, at runtime, it can't find the library that contains
>> _sift_features. Make sure that it's somewhere where your OS can find
>> it.
>
> It looks like the library implementing it was not linked into the
> extension. sift_features() is not part of OpenCV.
>
> James, are you including the source of Rob Hess's implementation with
> your extension, or are you trying to link against an already installed
> version of the library? If the former, you need to add the C sources
> to the pysift Extension(). If the latter, you need to add the name of
> the library to the list of libraries.

I'm including Rob Hess' sources with the extension.

Would that mean I should add library_dirs to Extension() to point to the
sources in the project's path?

> Also, you don't want to pass the list of libraries with
> extra_link_args. Instead, use libraries=.
>
> pysift = Extension('pysift',
>include_dirs = ['sift/include'],
>sources = ['src/pysift.c'],
>libraries = ['feat', 'cv', 'cxcore', 'highgui',
> 'cvaux', 'm'])

Thanks for taking a moment to help me out. :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: python extensions: including project local headers

[J Kenneth King]

Philip Semanchuk <[EMAIL PROTECTED]> writes:

> On Oct 23, 2008, at 11:36 AM, J Kenneth King wrote:
>
>>
>> Hey everyone,
>>
>> I'm working on a python extension wrapper around Rob Hess'
>> implementation of a SIFT feature detector. I'm working on a
>> computer-vision based project that requires interfacing with Python at
>> the higher layers, so I figured the best way to handle this would be
>> in
>> C (since my initial implementation in python was ungodly and slow).
>>
>> I can get distutils to compile the extension and install it in the
>> python path, but when I go to import it I get the wonderful exception:
>>
>> ImportError: /usr/lib/python2.5/site-packages/pysift.so: undefined
>> symbol: _sift_features
>
>
> Kenneth,
> You're close but not interpreting the error quite correctly. This
> isn't an error from the compiler or preprocessor, it's a library
> error. Assuming this is dynamically linked, your OS is reporting that,
> at runtime, it can't find the library that contains _sift_features.
> Make sure that it's somewhere where your OS can find it.

This is basically what I was looking for help with. So far the project
directory is:

/pysift
  /sift
..
/include
  ..
  sift.h
/src
  ..
  sift.c
  /src
pysift.c
  setup.py

I thought I could just #include "sift.h" in pysift.c as long as
distutils passed the right -I path to gcc.

Maybe I should compile the sift code as a shared object and link it to
my extension? How would I get distutils to build the makefile and tell
gcc how to link it?

Thanks for the reply. Python has spoiled me and my C is rather
rusty. :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: python extensions: including project local headers

[J Kenneth King]

Philip Semanchuk <[EMAIL PROTECTED]> writes:

> On Oct 23, 2008, at 3:18 PM, J Kenneth King wrote:
>
>> Philip Semanchuk <[EMAIL PROTECTED]> writes:
>>
>>> On Oct 23, 2008, at 11:36 AM, J Kenneth King wrote:
>>>
>>>>
>>>> Hey everyone,
>>>>
>>>> I'm working on a python extension wrapper around Rob Hess'
>>>> implementation of a SIFT feature detector. I'm working on a
>>>> computer-vision based project that requires interfacing with
>>>> Python at
>>>> the higher layers, so I figured the best way to handle this would be
>>>> in
>>>> C (since my initial implementation in python was ungodly and slow).
>>>>
>>>> I can get distutils to compile the extension and install it in the
>>>> python path, but when I go to import it I get the wonderful
>>>> exception:
>>>>
>>>> ImportError: /usr/lib/python2.5/site-packages/pysift.so: undefined
>>>> symbol: _sift_features
>>>
>>>
>>> Kenneth,
>>> You're close but not interpreting the error quite correctly. This
>>> isn't an error from the compiler or preprocessor, it's a library
>>> error. Assuming this is dynamically linked, your OS is reporting
>>> that,
>>> at runtime, it can't find the library that contains _sift_features.
>>> Make sure that it's somewhere where your OS can find it.
>>
>> This is basically what I was looking for help with. So far the project
>> directory is:
>>
>> /pysift
>>  /sift
>>..
>>/include
>>  ..
>>  sift.h
>>/src
>>  ..
>>  sift.c
>>  /src
>>pysift.c
>>  setup.py
>>
>> I thought I could just #include "sift.h" in pysift.c as long as
>> distutils passed the right -I path to gcc.
>
> That's true, and it sounds like you've got that part working.
>
>
>> Maybe I should compile the sift code as a shared object and link it to
>> my extension? How would I get distutils to build the makefile and tell
>> gcc how to link it?
>>
>> Thanks for the reply. Python has spoiled me and my C is rather
>> rusty. :)
>
> I don't know how to get setup.py to build a shared object separately.
> I am in the same Python/C situation as you. I'm scrubbing the rust off
> of my C skills and I'm also a n00b at developing extensions. I've
> learned a lot from looking at other people's setup code, so maybe I
> can help you there.
>
> My posix_ipc module links to the realtime lib "rt" and here's the
> relevant snippets of setup.py:
>
> --
> import distutils.core as duc
>
> libraries = [ ]
>
> libraries.append("rt")
>
> source_files = ["posix_ipc_module.c"]
>
> ext_modules = [ duc.Extension("posix_ipc",
>   source_files,
>   define_macros=define_macros,
>   libraries=libraries
>  )
>   ]
>
> duc.setup(name="posix_ipc", version=VERSION, ext_modules=ext_modules)
>
> --
>
> You can download the whole thing here if you want to examine all the
> code:
> http://semanchuk.com/philip/posix_ipc/
>
> HTH
> Philip

I'll take a look, thanks! :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python suitable for Midi ?

[J Kenneth King]

Derek Martin <[EMAIL PROTECTED]> writes:

> On Tue, Oct 28, 2008 at 06:54:57PM +0200, Chuckk Hubbard wrote:
>> The problem I've run into is that I can't set the audio to a higher
>> priority than the GUI (Tkinter).  If I move the mouse over the app, no
>> matter what, I get audio dropouts.  AFAICT this is the same for all
>> Python, regardless of what modules one uses: you can't assign system
>> priorities to different threads.  If you're planning to pipe MIDI to
>> another app for playback, maybe it won't be an issue for you.
>
> FWIW...  You could take your own advice, and devide your application
> in two: one process manages the GUI, and the second is a back-end
> process that plays the MIDI.  Your GUI can even launch the back end,
> which will inherit the priority of the GUI, after which the GUI can
> reduce its own priority (the priority of the back end will not be
> affected by the change)...
>
>
> -- 
> Derek D. Martin
> http://www.pizzashack.org/
> GPG Key ID: 0x81CFE75D

One also has access to nice-levels on unix systems.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Daemon and logging - the best approach?

[J Kenneth King]

Lech Karol Pawłaszek <[EMAIL PROTECTED]> writes:

> Hello.
>
> I'm trying to make a daemon and I want to log to a file its activity.
> I'm using logging module with a configuration file for it (loaded via
> fileConfig()).
>
> And now I want to read logging config file before daemonize the program
> because the file might not be accessible after daemonization. OTOH while
> daemonizing I am closing all opened file descriptors - including those
> opened for logging.
>
> What is the best approach to handle such situation? Should I close all
> FD before daemonizing or just forget it? Any other hints?
>
> Kind regards,

Depends.

For *NIX systems, it's a good idea to use the syslog daemon for logging
where it's available.

I believe the logging module can be configured to log to the local
syslog daemon.

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: best python unit testing framwork

[J Kenneth King]

Brendan Miller <[EMAIL PROTECTED]> writes:

> What would heavy python unit testers say is the best framework?
>
> I've seen a few mentions that maybe the built in unittest framework
> isn't that great. I've heard a couple of good things about py.test and
> nose. Are there other options? Is there any kind of concensus about
> the best, or at least how they stack up to each other?
>
> Brendan

I find nose to be the best. It's simple, easy, and doesn't sacrifice
power. All good things if you value your time. :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[J Kenneth King]

Joe Strout <[EMAIL PROTECTED]> writes:

> Let me preface this by saying that I think I "get" the concept of
> duck- 
> typing.
>
> However, I still want to sprinkle my code with assertions that, for
> example, my parameters are what they're supposed to be -- too often I
> mistakenly pass in something I didn't intend, and when that happens, I
> want the code to fail as early as possible, so I have the shortest
> possible path to track down the real bug.  Also, a sufficiently clever
> IDE could use my assertions to know the type of my identifiers, and so
> support me better with autocompletion and method tips.
>
> So I need functions to assert that a given identifier quacks like a
> string, or a number, or a sequence, or a mutable sequence, or a
> certain class, or so on.  (On the class check: I know about
> isinstance, but that's contrary to duck-typing -- what I would want
> that check to do instead is verify that whatever object I have, it has
> the same public (non-underscore) methods as the class I'm claiming.)
>
> Are there any standard methods or idioms for doing that?
>
> Thanks,
> - Joe

I generally use the 'assert' keyword when I'm in a rush otherwise unit
tests generally catch this kind of thing.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing vs. [Pyro, RPyC]

[J Kenneth King]

Jeffrey Barish <[EMAIL PROTECTED]> writes:

> [EMAIL PROTECTED] wrote:
>
>> 
>> Jeffrey> With the release of multiprocessing in Python 2.6, is there
>> any Jeffrey> reason to use Pyro or RPyC?
>> 
>> As far as I know the multiprocessing module only works on one machine
>> (multi-cpu or multi-core), not across machines.
>
> So I thought at first, but then I saw this statement in the documentation:
>
> It is possible to run a manager server on one machine and have clients use
> it from other machines (assuming that the firewalls involved allow it).

Depends. I don't know much about the multiprocessing module in 2.6,
but I have built a distributed application on Pyro.


Pyro has many advantages -- a query-able name server, GUI tools for
monitoring your setup, and remote agents. It is also rather simple in
comparison to other similar tools (*cough*twisted.pb*cough*). However,
it is essentially an RPC style system, so some people might not be too
comfortable with it. YMMV.
--
http://mail.python.org/mailman/listinfo/python-list

function parameter scope python 2.5.2

[J Kenneth King]

I recently encountered some interesting behaviour that looks like a bug
to me, but I can't find the appropriate reference to any specifications
to clarify whether it is a bug.

Here's the example code to demonstrate the issue:

class SomeObject(object):

def __init__(self):
self.words = ['one', 'two', 'three', 'four', 'five']

def main(self):
recursive_func(self.words)
print self.words

def recursive_func(words):
if len(words) > 0:
word = words.pop()
print "Popped: %s" % word
recursive_func(words)
else:
print "Done"

if __name__ == '__main__':
weird_obj = SomeObject()
weird_obj.main()


The output is:

Popped: five
Popped: four
Popped: three
Popped: two
Popped: one
Done
[]

Of course I expected that recursive_func() would receive a copy of
weird_obj.words but it appears to happily modify the object.

Of course a work around is to explicitly create a copy of the object
property befor passing it to recursive_func, but if it's used more than
once inside various parts of the class that could get messy.

Any thoughts? Am I crazy and this is supposed to be the way python works?
--
http://mail.python.org/mailman/listinfo/python-list

Re: function parameter scope python 2.5.2

[J Kenneth King]

J Kenneth King <[EMAIL PROTECTED]> writes:

> I recently encountered some interesting behaviour that looks like a bug
> to me, but I can't find the appropriate reference to any specifications
> to clarify whether it is a bug.
>
> Here's the example code to demonstrate the issue:
>
> class SomeObject(object):
>
> def __init__(self):
> self.words = ['one', 'two', 'three', 'four', 'five']
>
> def main(self):
> recursive_func(self.words)
> print self.words
>
> def recursive_func(words):
> if len(words) > 0:
> word = words.pop()
> print "Popped: %s" % word
> recursive_func(words)
> else:
> print "Done"
>
> if __name__ == '__main__':
> weird_obj = SomeObject()
> weird_obj.main()
>
>
> The output is:
>
> Popped: five
> Popped: four
> Popped: three
> Popped: two
> Popped: one
> Done
> []
>
> Of course I expected that recursive_func() would receive a copy of
> weird_obj.words but it appears to happily modify the object.
>
> Of course a work around is to explicitly create a copy of the object
> property befor passing it to recursive_func, but if it's used more than
> once inside various parts of the class that could get messy.
>
> Any thoughts? Am I crazy and this is supposed to be the way python works?

Of course, providing a shallow (or deep as necessary) copy makes it
work, I'm curious as to why the value passed as a parameter to a
function outside the class is passed a reference rather than a copy.
--
http://mail.python.org/mailman/listinfo/python-list

Re: function parameter scope python 2.5.2

[J Kenneth King]

alex23 <[EMAIL PROTECTED]> writes:

> On Nov 21, 9:40 am, J Kenneth King <[EMAIL PROTECTED]> wrote:
>> Of course, providing a shallow (or deep as necessary) copy makes it
>> work, I'm curious as to why the value passed as a parameter to a
>> function outside the class is passed a reference rather than a copy.
>
> You're passing neither a reference nor a copy, you're passing the
> object (in this case a list) directly:
>
> http://effbot.org/zone/call-by-object.htm

Ah, thanks -- that's precisely what I was looking for.

I knew it couldn't be a mistake; I just couldn't find the documentation
on the behaviour since I didn't know what it was called in the python
world.

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: function parameter scope python 2.5.2

[J Kenneth King]

Steven D'Aprano <[EMAIL PROTECTED]> writes:

> On Thu, 20 Nov 2008 18:31:12 -0500, J Kenneth King wrote:
>
>> Of course I expected that recursive_func() would receive a copy of
>> weird_obj.words but it appears to happily modify the object.
>
> I am curious why you thought that. What made you think Python should/did 
> make a copy of weird_obj.words when you pass it to a function?
>
> This is a serious question, I'm not trying to trap you into something :)

Don't worry, I don't feel "trapped" in usenet. ;)

It was more of an intuitive expectation than a suggestion that Python
got something wrong.

I was working on a program of some complexity recently and quickly
caught the issue in my tests. I knew what was going on and fixed it
expediently, but the behaviour confused me and I couldn't find any
technical documentation on it so I figured I just didn't know what it
was referred to in Python. Hence the post. :)

I suppose I have some functional sensibilities and assumed that an
object wouldn't let a non-member modify its properties even if they were
mutable.

Of course if there is any further reading on the subject, I'd appreciate
some links.

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

Re: function parameter scope python 2.5.2

[J Kenneth King]

Peter Pearson <[EMAIL PROTECTED]> writes:

> On Fri, 21 Nov 2008 10:12:08 -0500, J Kenneth King wrote:
>> Steven D'Aprano <[EMAIL PROTECTED]> writes:
>>>
>>> I am curious why you thought that. What made you think Python should/did 
>>> make a copy of weird_obj.words when you pass it to a function?
> [snip]
>> Of course if there is any further reading on the subject, I'd appreciate
>> some links.
>
> As one relatively new Python fan to another, I recommend following
> this newsgroup.  Many important aspects of Python that several books
> failed to drive through my skull are very clearly (and repeatedly)
> explained here.  Hang around for a week, paying attention to posts
> with subjects like "Error in Python subscripts" (made-up example),
> and curse me if you don't find it greatly rewarding.

I do lurk more often than I post and sometimes I help out people new to
Python or new to programming in general. I know how helpful usenet can
be and usually this group in particular is quite special. It's good
advice to read before you post; quite often the question has been
proposed and answered long before it came to your little head (not you
in particular; just general "you").

In this case, I was simply lacking the terminology to find what I was
looking for on the subject. In such cases turning to the community seems
like a fairly reasonable way to find clarification. I've only been
programming in Python specifically for two years or so now, so I hope I
can be forgiven.

Cheers.
--
http://mail.python.org/mailman/listinfo/python-list

ANN: tracshell 0.1r23

[J Kenneth King]

Hello everyone,

Just wanted to let everyone know that I'm still working on TracShell and
it has come a long way in the last couple of weeks!

As of r23, TracShell has introduced the following features and fixes:

- TracShell now queries the Trac server for a list of available methods
  and ensures your shell session only has the corresponding commands.
- Command shortcuts (`help shortcuts` to see a list)
- SSL support
- A slew of bugfixes

Just head on over to the project page and grab a fresh copy from the
repository: http://code.google.com/p/tracshell

Thanks to the contributors who've already helped a lot by submitting
patches.

Bug reports, feature requests, comments and critiques are encouraged!

All the best,

J Kenneth King
--
http://mail.python.org/mailman/listinfo/python-list

Re: While loop

[J Kenneth King]

Fab86  writes:

> On Mar 5, 5:23 pm, Marco Mariani  wrote:
>> Fab86 wrote:
>> > Is it possible to get the program to catch the exception, wait 10
>> > seconds, then carry of from where it was rather than starting again?
>>
>> something like this? probably works in PASCAL as well :)
>>
>> > i=0
>> > while i < len(stuff):
>> >    try:
>> >       do_with(stuff[i])
>> >    except SomeError:
>> >       sleep(10)
>> >       continue
>> >    i+=1
>>
>>
>
> using sleep and then continue just makes the search start from the
> first search term like before.. Would it be easier to understand if I
> posted sections of my code?
>
> i = 0
> k = open('blah', 'w')
> domains = ["au", "ca", "nl", "be", "...]
> srch = WebSearch(app_id=YahooKey)
>
> while i<200:
> try:
> for domain in domains:
> srch.query = "test site:.%s" % domain
> res = srch.parse_results()
> print >> k, res.total_results_available
> i = i + 1
>
> except SearchError:
>
> (I currently close then reopen document here then restart i to 0)
>
> Any ideas?

You should check out the sched module for scheduling tasks.



import sched, time

domains = ["au", "ca", "nl", "be", "ru", "us", "de"]
s = sched.scheduler(time.time, time.time)

def do_query(query, domain):
search.query = "%s site:.%s" % (query, domain)
try:
result = search.parse_results()
except SearchError, e:
print >> sys.stderr, e
else:
print >> k, result

for domain in domains:
s.enter(2, 1, do_query, domain)

s.run()



YMMV
--
http://mail.python.org/mailman/listinfo/python-list

Re: Roundup Issue Tracker release 1.4.7

[J Kenneth King]

Richard Jones  writes:

> I'm proud to release version 1.4.7 of Roundup.

> - Allow CGI frontend to serve XMLRPC requests.
> - Added XMLRPC actions, as well as bridging CGI actions to XMLRPC actions.

Sweet.

I'm working on a small project called TracShell which is a command-line
front-end to the Trac issue tracker and uses XML-RPC to communicate with
the server.

However, as I work with it/hack on it I'm slowly building it into a
generic front-end to work with any XML-RPC enabled issue tracker.

I'll definitely look into extending it to work with Roundup in the
future now that it has XML-RPC support. :)

Keep up the great work!
--
http://mail.python.org/mailman/listinfo/python-list

mocking xmlrpclib

[J Kenneth King]

At the risk of sounding like I don't know what I'm doing, I must say
that I am finding it rather difficult/tedious to mock the xmlrpclib
interface using minimock.

I refuse to believe that I'm the only developer to have tried this
before, but google isn't being my friend and I can't seem to get it to
work outside the interpreter (ie: in actual unit tests).

Has anyone in c.l.p encountered this problem or have any tips?
--
http://mail.python.org/mailman/listinfo/python-list

Re: HTML Generation

[J Kenneth King]

Tino Wildenhain  writes:

> Hi Mike,
>
>
> Mike wrote:
>> Hello all,
>>
>> I'm writing a web app and wanted to do some html generation (I
>> really do not like to maintain or write html).
>>
>> I'm thinking of writing a dsl based on the following:
>>
>> def html():
>> return
>>
>> def a():
>> return
>>
>> def body():
>> return
>
> That would be writing HTML just another way and also
> mixing code and representation, this is generally not a
> good idea.
>
> If you really don't want to maintain HTML along with
> your code, I'd suggest an approach like
>
> Zope Page Templates
>
> http://en.wikipedia.org/wiki/Zope_Page_Templates#Zope_Page_Templates
>
> which can be used outside Zope as well, (even with many other languages)
>
> For example: http://zpt.sourceforge.net/
>
> Regards
> Tino

You could also look at CL-WHO for inspiration.

I feel the same way about templates -- the seperation is unnecessary
complexity. One knowledge domain for everything please. We're not
building the tower of Babel here.

I use Python through all of my desktop application development without
switching to markup languages and stylesheets and other scripting
languages (at least VERY rarely).

Why is the web any different?

It's rather somewhat trivial to implement a simple grammar in Python for
describing a page using Python data structures and Pythonic idioms.

I've toyed with it using a base "Tag" metaclass which creates the
__str__() representation for a Tag-metaclassed object based on the name
of the class.

It would look something like this:

code:


from tags import html, head, meta, title, body, div, p, a

mypage = html(
 head(
 meta(attrs={'http-equiv': "Content-Type",
 'content': "text/html;"}),
 title("My Page")),
 body(attrs={'background': "#ff;"},
 div(attrs={'id': "article-content"},
 p(a(attrs={'href': "http://python.org"},
"Python.org")

tabbed_file(mypage, open('myfile.html', 'w'))


Unfortunately Python isn't as malleable when it comes to syntax as Lisp,
but I don't think it's a lost endeavour.

Having to know a template language, markup, stylesheets, CSS, plus all
of Python and good web application design is a huge headache and puts
too many cooks in the kitchen. I'd rather 95% of the work be in pure
Python and keep the team small.

Just 0.02 monetary units.
--
http://mail.python.org/mailman/listinfo/python-list

Re: HTML Generation

[J Kenneth King]

Stefan Behnel  writes:

> J Kenneth King wrote:
>> from tags import html, head, meta, title, body, div, p, a
>> 
>> mypage = html(
>>  head(
>>  meta(attrs={'http-equiv': "Content-Type",
>>  'content': "text/html;"}),
>>  title("My Page")),
>>  body(attrs={'background': "#ff;"},
>>  div(attrs={'id': "article-content"},
>>  p(a(attrs={'href': "http://python.org"},
>> "Python.org")
>> 
>> tabbed_file(mypage, open('myfile.html', 'w'))
>
> See here for another example that uses lxml.html:
>
> http://codespeak.net/lxml/lxmlhtml.html#creating-html-with-the-e-factory
>
> Stefan

Ah, looks good. Have never used nor finished the example I had given --
only meant as inspiration. I'm not surprised it has been done by someone
else.

I've been thinking about the co-routines presentation recently given at
Pycon and have been thinking about ways they could be used to extend the
grammar a bit.

At present, it looks like in my example and the lxml.html example that
the HTML structures created are declarative. Scripting the generation of
child elements for example would break up the flow...

code:


E.HTML(
E.HEAD(
E.TITLE("Best Page Evar!")
),
E.BODY(
E.H1(E.CLASS("best-articles-heading"), "Best Articles"),
E.UL(E.CLASS("articles"))
)
)

for article in articles:
E.HTML.BODY.UL.append(E.LI(article['title]))


... at least that's how I assume it would work.

I think in my prototype you could use list-comprehensions in-line to add
the elements, but it was quite an eyesore.

code:


my_html = html(
  head(
  title("Best Page Evar!")
  ),
  body(
  ul(
  [li(article['title']) for article in articles]
  )
  )
  )


I guess it's not that bad, but I had a feeling it could look prettier
like how naturally CL-WHO reads in Lisp. It's just that the list
comprehension doesn't read well in this context; it'd be more natural to
read it as "for each article in article, create a list element with the
article title" instead.

I get the impression that creating a chain of co-routines would reveal a
grammar for talking about generating web pages. Something like...

code:


html_to_file(
html(
head(
title("Best page evar")
),
body(
h1({'class': "underline-heading"},
   "Newest Articles"),
unordered_list(
articles,
['title', 'href'],
'%-10s: %s')
)
),
file=sys.stdout
)


Where `unordered_list` would walk the elements in its first argument,
extract the values from the keys specified by its second argument, and
format the results using the string from its third argument and simply
return a ul() object with nested li() objects with all the data inserted
into them.

Of course, this is very off-the-cuff; I only started picking up interest
in this old subject this morning. ;) I could be talking way out of my
ass here. No idea if any of it's even practical.

Anyway -- OP: there are many ways to approach HTML generation and it's a
good pursuit. If you come up with something new and unique, please
share! Down with templates! :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: decorators don't play nice with nose?

[J Kenneth King]

hyperboreean  writes:

> From: hyperboreean 
> Subject: decorators don't play nice with nose?
> Newsgroups: comp.lang.python
> To: python-list@python.org
> Date: Mon, 06 Apr 2009 11:01:04 +0300
>
> Hi, I am trying to test the business part of a web service. For this I
> am using unittest & nose.
> I wrote a decorator that should handle the xml test file retrieval,
> but it seems I can't get it working with nose.
> Here's the code:
>
>
> * MyApp.py -- base test class *
>
> import os
> import unittest
>
> from MyApp.Core import XmlParser
>
>
> __all__ = ['MyAppTest', 'setup']
>
>
> PATH = os.path.dirname(__file__) or ''
>
>
> class setup(object):
>"""Decorator to ease the use of xml files in MyApp tests.
>
>The way it works it that it decorates a test method which has a first
>default parameter called 'parser' and it overwrites this parameter value
>with a XmlParser instance.
>
>The xml file should be located under:
>data/testedBusinessRequest/testMethodName.xml
>"""
>def __init__(self, testedBusinessRequest = ''):
>self.testedBusinessRequest =\
>testedBusinessRequest.lower()
>
>
>def _getXmlParser(self, xml):
>documentElement = XmlParser.parseXmlStream(xml)
>parser = XmlParser.getParser(documentElement)
>return parser
>
>
>def __call__(self, method):
>
># TODO: error handling here
>methodName = method.func_code.co_name
>methodName = methodName.split('_')[1]
>
>xmlFolder = self.testedBusinessRequest
>xmlFile = '%s.xml' % methodName
>
>path = os.path.join(PATH, 'data',
>xmlFolder, xmlFile)
>
>f = open(path)
>xml = f.read()
>f.close()
>method.func_defaults = (self._getXmlParser(xml),)
>return method
>
>
> class MyAppTest(unittest.TestCase):
>
>def setUp(self):
>self.database = Database()
>
>def tearDown(self):
>pass
>
>
> * test_Login.py - test a business request *
> from MyAppTest import MyAppTest, setup
>
> from MyApp import Login
>
>
> class TestLogin(MyAppTest):
>testedBusinessRequest = 'Login'
>
>@setup(testedBusinessRequest)
>def test_validParameters(self, parser = None):

FWIW, nose and unittest both provide methods for setting up and
tearing down tests. You should probably look at those first before
rolling your own. At the very least it will give you an idea of how
yours should work.

Cheers
--
http://mail.python.org/mailman/listinfo/python-list

unpythonic use of property()?

[J Kenneth King]

Consider:

code:


class MyInterface(object):

def __get_id(self):
return self.__id

id = property(fget=__get_id)

def __init__(self, id, foo):
self.__id = id
self.foo = foo


class MyInterface2(object):

def __init__(self, id, foo):
self._id = id
self.foo = foo

@property
def id(self):
return self._id



The situation is that an API to an external resource must enforce
consistency with the resource's expectations of data (ie: ID's must be
immutable).

The Python docs clearly show the use of the property() decorator in both
the example classes shown here. Both ensure that an assignment to the
class objects' 'id' attribute will raise an AttributeError
exception. Both will satisfy the requirements.

I assume we all know what the implementation differences are between the
two examples here. What I am curious about is what use case would the
MyInterface example have? My guess is that it's added protection for
attribute names on objects generated from factories or generators. I'd
like a more concrete explanation from someone who has encountered a
scenario where it was required.

I was recently informed that it was 'unpythonic' and have since been a
little confused by the term. I've heard it bandied about before but
never paid much attention. What is 'unpythonic'? What about this example
is unpythonic?
--
http://mail.python.org/mailman/listinfo/python-list

Re: unpythonic use of property()?

[J Kenneth King]

Carl Banks  writes:

> On Apr 17, 4:00 pm, Scott David Daniels  wrote:
>> Carl Banks wrote:
>> > On Apr 17, 10:21 am, J Kenneth King  wrote:
>> >> Consider:
>>
>> >> code:
>> >> 
>>
>> >> class MyInterface(object):
>>
>> >>     def __get_id(self):
>> >>         return self.__id
>>
>> >>     id = property(fget=__get_id)
>>
>> >>     def __init__(self, id, foo):
>> >>         self.__id = id
>> >>         self.foo = foo
>>
>> >> class MyInterface2(object):
>>
>> >>     def __init__(self, id, foo):
>> >>         self._id = id
>> >>         self.foo = foo
>>
>> >>     @property
>> >>     def id(self):
>> >>         return self._id
>>
>> ...
>> >> I was recently informed that it was 'unpythonic' and have since been a
>> >> little confused by the term. I've heard it bandied about before but
>> >> never paid much attention. What is 'unpythonic'? What about this example
>> >> is unpythonic?
>>
>> > There are different reasons someone might have said it.
>> > ...
>> > Some people think attribute name-mangling is unpythonic.  It's true
>> > that people sometimes mistakenly treat it a solid information hiding
>> > mechanism, but I wouldn't call its usage unpythonic when used as
>> > intended: as a way to avoid name-collisions.  If you think it's
>> > worthwhile to protect an attribute from being overwritten, you might
>> > as well guard against accidental conflict with the underlying name.
>>
>> Here you are assuming that a user of your class could not possibly have a
>> valid reason for getting to the underlying variable.  Don't make those
>> decisions for someone else, in Python, "we are all adults here."
>
> They can use the demangled name of the internal variable if they want
> access to it.
>
>
>> > Finally, some people think read-only attributes are unpythonic.  I
>> > think that's ridiculous, although in general I'd advise against making
>> > attributes read-only willy-nilly.  But there's a time and place for
>> > it.
>>
>> Generally, properties are for doing some form of calculation, not
>> for making things read-only.
>
> That might be how properties are "generally" used, but if for some
> reason I wanted a read-only attribute, that's how I'd do it.
>
>
> [snip strawman stuff]
>> It is not
>> your job to protect those users who do not use your code properly from
>> themselves; that way lies madness.
>
> I'm sorry, but the universe is not as simple as you are making it out
> to be.  Blanket statements like the one you just gave here are not
> something that should ever be blindly adhered to.
>
> If, in my judgment, users would be prone to overwrite one of my
> attributes, and if I designed the system to rely on that attribute,
> and if the results of changing it are bad enough, then by golly I'm
> going to make the attribute harder than usual to modify.  And yes,
> that is my job.
>
> Users who want to change it anyway can curse me and then go demangle
> the name themselves.
>
>
> Carl Banks

Thanks for all the replies --

While greatly simplified, the reason for desiring read-only attributes
in this case is that the interface is to a remote server whose policy
relies on data objects represented by this class to have a few values
which never change or are managed by the server and not the end
user. Changing the ID value would break things on the server, so I
wanted to write the interface class to respect those conventions.

I'm well aware that if a developer really wanted to, they could get
around it no matter what I did, but I figure that if I at least make
it really difficult it will be obvious that they're really going into
dangerous territory.

Further (and this might just be a tad paranoid), user interface code
which might use this API might be dangerous. It's one thing for a
developer to break the rules when they need to, but a user shouldn't
be able to. By enforcing read-only on the API it ensure (at least in
my world view) that a developer writing a user interface against it
won't have to code defensively against malicious input.

However, since the difference between the two is simply attribute
name-mangling it's practically a pedantic issue. I guess there might
be some hyper-specific scenario where MyInterface would still be
useful, but this one might not be it.

Again, thanks. :)
--
http://mail.python.org/mailman/listinfo/python-list

Re: unpythonic use of property()?

[J Kenneth King]

Luis Zarrabeitia  writes:

> On Monday 20 April 2009 11:29:19 am J Kenneth King wrote:
>> Changing the ID value would break things on the server, so I
>> wanted to write the interface class to respect those conventions.
>
> Then, take this opportunity fix the server and prevent it from breaking once 
> you change the ID, because:

Unfortunately it's not my server to fix. I can suggest a patch, but
that's it.

>
>> I'm well aware that if a developer really wanted to, they could get
>> around it no matter what I did, but I figure that if I at least make
>> it really difficult it will be obvious that they're really going into
>> dangerous territory.
>
> you will have to do it anyway.
>
> I think it's great, for you, that the language you are using makes it so 
> extremely easy to bypass almost any access restriction that you may put in 
> the data sent to your clients. That means that you won't be overconfident, 
> that you are forced to make sound decisions from the beginning, and that you 
> won't be trusting that your clients will be well behaved (even with very 
> strong enforcement of data hiding, a malicious client could forge the 
> connection).
>
> Then, when the server is safe from user data, by all means go and make sure 
> that the clients (and even the server!) will complain as loudly as possible 
> if they want to break encapsulation.

Well, really all I'm doing is writing a light framework around xmlrpclib.

The design allows me to keep this kind of exception highly localized,
so its not really a huge practical issue.

The problem I had was that I found two approaches to the same problem
that appeared to be pedantic in difference.

However, you are right of course -- logically the server should
protect against this vulnerability itself instead of the
implementation using my framework.
--
http://mail.python.org/mailman/listinfo/python-list

Re: unpythonic use of property()?

[J Kenneth King]

Luis Zarrabeitia  writes:

> On Wednesday 22 April 2009 01:44:38 pm J Kenneth King wrote:
>> > Then, take this opportunity fix the server and prevent it from breaking
>> > once you change the ID, because:
>>
>> Unfortunately it's not my server to fix. I can suggest a patch, but
>> that's it.
>
> Yes, that's unfortunate.
> Btw, when I re-read my phrase by itself, it seemed hostile... My apologies. 
> I'm still not very good at expressing my thoughts in english.

It's okay.

You have great English. ;)

>
> Then, I guess, you have little choice. Mangle the name, hope that the server 
> will get fixed.
>
> -- 
> Luis Zarrabeitia (aka Kyrie)
> Fac. de Matemática y Computación, UH.
> http://profesores.matcom.uh.cu/~kyrie
--
http://mail.python.org/mailman/listinfo/python-list

Re: python list handling and Lisp list handling

[J Kenneth King]

Steven D'Aprano  writes:

> On Sat, 25 Apr 2009 23:51:18 -0700, namekuseijin wrote:
>
>> On Apr 26, 1:31 am, Steven D'Aprano > cybersource.com.au> wrote:
>>> On Fri, 24 Apr 2009 21:01:10 -0700, Carl Banks wrote:
>>> > That's because Python lists aren't lists.
>>>
>>> Surely you meant to say that Lisp lists aren't lists?
>>>
>>> It-all-depends-on-how-you-define-lists-ly y'rs,
>> 
>> Yeah, the List Processing language got it all wrong by not going with
>> arrays like Python...
>
> Well, Lisp was invented in 1958, before anyone knew how to program *wink*.

And 50+ years of development hasn't taught them anything. :p

Guess you don't know anything about programming unless you're new...

> Seriously though, linked lists are not the only sort of list. That was my 
> point: first define what is a list, and then we can debate what is or 
> isn't a list. Even within linked lists, there are various different 
> types, all with their own strengths and weaknesses: singly-linked lists, 
> doubly-linked lists, circular lists, open lists, xor-lists, lists with or 
> without sentinels, lists with internal and external storage, unrolled 
> linked lists, and combinations of all of the above.

And any sufficiently powerful language would allow the programmer to
adapt to any type they needed. ;)

Interesting topic though.
--
http://mail.python.org/mailman/listinfo/python-list

Re: list comprehension question

[J Kenneth King]

Chris Rebert  writes:

> On Thu, Apr 30, 2009 at 5:56 PM, Ross  wrote:
>> If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
>> return a new list of each individual element in these tuples, I can do
>> it with a nested for loop but when I try to do it using the list
>> comprehension b = [j for j in i for i in a], my output is b =
>> [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
>> doing wrong?
>
> Your comprehension is the identity comprehension (i.e. it effectively
> just copies the list as-is).
> What you're trying to do is difficult if not impossible to do as a
> comprehension.
>
> Here's another approach:
> b = list(itertools.chain.from_iterable(a))
>
> And without using a library function:
> b = []
> for pair in a:
> for item in pair:
> b.append(item)

This is much more clear than a nested comprehension.

I love comprehensions, but abusing them can lead to really dense and
difficult to read code.
--
http://mail.python.org/mailman/listinfo/python-list

Re: list comprehension question

[J Kenneth King]

Emile van Sebille  writes:

> On 5/1/2009 7:31 AM J Kenneth King said...
>> Chris Rebert  writes:
>>> b = []
>>> for pair in a:
>>> for item in pair:
>>> b.append(item)
>>
>> This is much more clear than a nested comprehension.
>>
>> I love comprehensions, but abusing them can lead to really dense and
>> difficult to read code.
>
> I disagree on dense and difficult, although I'll leave open the
> question of abuse.

Dense and difficult may be subjective to people like you or I, but you
left out the "to read" part of that sentence. I was referring to the
negative effect nested or complex list comprehensions can have on
readability.

> b = [ item for pair in a for item in pair ]

It's a clever statement, but use it once and its gone. Why not use the
expanded form in a function definition and get an even shorter, but
clearer statement?

>>> b = flatten(a)

Boom, done.

> This is exactly the code above expressed in comprehension form.
>
> It's worth knowing that a list comprehension is structured identically
> to the equivalent for loop.  So it really is neither more dense nor
> more difficult to read.  Further, you can tell immediately from the
> start of the list comprehension what you've got -- in this case a list
> of item(s).
>
> Here with some slight changes...
>
>>>> a = [(1, 2), (3, 4, 7), (5, 6)]
>>>> [ item for j in a if len(j)==2 for item in j if item % 2 ]
> [1, 5]
>
> ...opposed to...
>
>>>> for j in a:
> ... if len(j)==2:
> ... for item in j:
> ... if item % 2:
> ... b.append(item)
> ...
>>>> b
> [1, 5]
>>>>

Thanks for the lesson in list comprehensions, but I'm well aware of
how they work.

List comprehensions can make a reader of your code apprehensive
because it can read like a run-on sentence and thus be difficult to
parse. The Python documentation discourages their use and I believe
for good reason. It's much easier to explain complex processes with a
function rather than a single nested statement.

>
> YMMV,
>
> Emile

It will apparently vary greatly. Depending on how much coffee I've
had.

;)

J
--
http://mail.python.org/mailman/listinfo/python-list

Re: Any idea to emulate tail -f

[J Kenneth King]

Iain King  writes:

> On May 5, 7:00 am, Joel Juvenal Rivera Rivera 
> wrote:
>> I want to make something very similar to  the command tail -f (follow a
>> file), i have been trying  with some while True and some microsleeps
>> (about .1 s); did someone has already done something like this?
>>
>> And about the file is the apache acceslog  of a site with a lot of
>> traffic.
>>
>> Regards    
>>
>> joe / Joel Rivera
>
> This is very interesting, about using Generator Expressions:
> http://209.85.229.132/search?q=cache:ZHrV4E0eTI8J:www.dabeaz.com/generators/Generators.pdf
>
> Relevant stuff about 'tail -f' is on page 39, but I'd read the whole
> thing if you can.
>
> Iain

+1

I second this suggestion. I've used this method in many scripts of
late and it is quite handy.

He even takes it a little further and shows you some neat things you
can do with it later on.
--
http://mail.python.org/mailman/listinfo/python-list

Re: list comprehension question

[J Kenneth King]

Emile van Sebille  writes:

> On 5/5/2009 9:15 AM J Kenneth King said...
>
>> List comprehensions can make a reader of your code apprehensive
>> because it can read like a run-on sentence and thus be difficult to
>> parse. The Python documentation discourages their use and I believe
>> for good reason.
>
> Can you provide a link for this?  I'd like to see specifically what's
> being discouraged, as I'd be surprised to find routine usage frowned
> upon.
>
> Emile

http://docs.python.org/tutorial/datastructures.html#nested-list-comprehensions


"If you’ve got the stomach for it, list comprehensions can be
nested. They are a powerful tool but – like all powerful tools – they
need to be used carefully, if at all."

and

"In real world, you should prefer builtin functions to complex flow
statements."
--
http://mail.python.org/mailman/listinfo/python-list

Re: list comprehension question

[J Kenneth King]

Steven D'Aprano  writes:

> On Wed, 06 May 2009 09:48:51 -0400, J Kenneth King wrote:
>
>> Emile van Sebille  writes:
>> 
>>> On 5/5/2009 9:15 AM J Kenneth King said...
>>>
>>>> List comprehensions can make a reader of your code apprehensive
>>>> because it can read like a run-on sentence and thus be difficult to
>>>> parse. The Python documentation discourages their use and I believe
>>>> for good reason.
>>>
>>> Can you provide a link for this?  I'd like to see specifically what's
>>> being discouraged, as I'd be surprised to find routine usage frowned
>>> upon.
>>>
>>> Emile
>> 
>> http://docs.python.org/tutorial/datastructures.html#nested-list-
> comprehensions
>> 
>> 
>> "If you’ve got the stomach for it, list comprehensions can be nested.
>> They are a powerful tool but – like all powerful tools – they need to be
>> used carefully, if at all."
>
> How does this discourage the use of list comprehensions? At most, it 
> warns that complicated list comps are tricky. Complicated *anything* are 
> tricky.

They are tricky and need to be used carefully, *if at all*.

IMO this means that if there's a way to do it without a nested list
comprehension, then that solution should be preferred.

>> and
>> 
>> "In real world, you should prefer builtin functions to complex flow
>> statements."
>
> That's ridiculous. The example given is a special case. That's like 
> saying "Loops are hard, so in the real world, if you want a loop, find a 
> builtin function that does what you want instead."
>
> What's the "builtin function" we're supposed to prefer over a "complex 
> flow statement" like this?

It's not ridiculous and says nothing of the sort.  You're jumping to
conclusions.  If you have a problem with it, I suggest you complain to
the Python documentation people.  I don't make this stuff up.

Just look at many of the example solutions provided in this thread to
the OP's problem. The examples in the link I provided are also good as
are ones provided in the itertools documentation.

Keep in mind that nested comprehensions are still available because
they do have a use case that justifies their existence. However, I
think the Python documentation warns against their use because people
might rely on them for problems where they aren't necessary and since
they are difficult to read... it can lead to difficult to read code.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Best practice for operations on streams of text

[J Kenneth King]

James  writes:

> Hello all,
> I'm working on some NLP code - what I'm doing is passing a large
> number of tokens through a number of filtering / processing steps.
>
> The filters take a token as input, and may or may not yield a token as
> a result. For example, I might have filters which lowercases the
> input, filter out boring words and filter out duplicates chained
> together.
>
> I originally had code like this:
> for t0 in token_stream:
>   for t1 in lowercase_token(t0):
> for t2 in remove_boring(t1):
>   for t3 in remove_dupes(t2):
> yield t3
>
> Apart from being ugly as sin, I only get one token out as
> StopIteration is raised before the whole token stream is consumed.
>
> Any suggestions on an elegant way to chain together a bunch of
> generators, with processing steps in between?
>
> Thanks,
> James

Co-routines my friends. Google will help you greatly in discovering
this processing wonder.
--
http://mail.python.org/mailman/listinfo/python-list

Re: list comprehension question

[J Kenneth King]

Terry Reedy  writes:

> J Kenneth King wrote:
>>
>> Keep in mind that nested comprehensions are still available because
>> they do have a use case that justifies their existence.
>
> Nested comprehensions are available because because the syntax makes
> them available by default and making a fiddly exception would be
> contrary to Python's style.  A list comp creates an iterable. A list
> comp requires use of an iterable.  Therefore, a list comp may use a
> list comp.
>
>> However, I
>> think the Python documentation warns against their use because people
>> might rely on them for problems where they aren't necessary and since
>> they are difficult to read... it can lead to difficult to read code.
>
> Whenever the expression that results in the iterable used by a list
> comp is sufficiently complex, readability is improved by pulling it
> out as a separate statement. Nested list comps are often examplex of
> such sufficiently complex expressions, but not the only possible one.
>
> tjr

Agreed. A very succinct explanation of the point I was trying to make.
--
http://mail.python.org/mailman/listinfo/python-list

Re: list comprehension question

[J Kenneth King]

Steven D'Aprano  writes:

> On Thu, 07 May 2009 13:28:10 -0400, J Kenneth King wrote:
>
>> Steven D'Aprano  writes:
>> 
>>> On Wed, 06 May 2009 09:48:51 -0400, J Kenneth King wrote:
>>>
>>>> Emile van Sebille  writes:
>>>> 
>>>>> On 5/5/2009 9:15 AM J Kenneth King said...
>>>>>
>>>>>> List comprehensions can make a reader of your code apprehensive
>>>>>> because it can read like a run-on sentence and thus be difficult to
>>>>>> parse. The Python documentation discourages their use and I believe
>>>>>> for good reason.
>>>>>
>>>>> Can you provide a link for this?  I'd like to see specifically what's
>>>>> being discouraged, as I'd be surprised to find routine usage frowned
>>>>> upon.
>>>>>
>>>>> Emile
>>>> 
>>>> http://docs.python.org/tutorial/datastructures.html#nested-list-
>>> comprehensions
>>>> 
>>>> 
>>>> "If you’ve got the stomach for it, list comprehensions can be nested.
>>>> They are a powerful tool but – like all powerful tools – they need to
>>>> be used carefully, if at all."
>>>
>>> How does this discourage the use of list comprehensions? At most, it
>>> warns that complicated list comps are tricky. Complicated *anything*
>>> are tricky.
>> 
>> They are tricky and need to be used carefully, *if at all*.
>> 
>> IMO this means that if there's a way to do it without a nested list
>> comprehension, then that solution should be preferred.
>
> Earlier, you claimed that list comps in general were discouraged:
>
> "List comprehensions can make a reader of your code apprehensive because 
> it can read like a run-on sentence and thus be difficult to parse. The 
> Python documentation discourages their use and I believe for good reason."

Ooops. Typo. My bad. Had it been quoted earlier it would have saved a
few posts for sure. Such is the Internet. :)

> Emile said "I'd be surprised to find routine usage frowned upon", giving 
> you the opportunity to correct his (mis)understanding. You failed to do 
> so, which I took as meaning that you agreed that routine usage of simple 
> list comps were frowned upon. Now you're talking about *nested* list 
> comps. You started off (apparently) objecting to list comps in general, 
> because they "can" make readers apprehensive. Now you seem to be saying 
> that it's only the complicated, overly-dense ones which rightly make 
> readers apprehensive which you object to.

I was rather confused by that. I use list comps all the time and said
several times that I don't object to list comps, just nested ones.

> I suppose we're making progress if we agree that the Python docs warn 
> against unnecessarily complicated nested list comps. Whether it 
> discourages as well as warns is a matter of interpretation. But there's 
> certainly no sign that list comps as a general technique are discouraged 
> just because overly-complicated list comps are tricky to read. The same 
> can be said about *any* piece of code.

Well I did my best to make my interpretation clear. If the
documentation says that nested list comps are difficult to read and
should be used rarely, if at all, then I generally consider that
"discouraging" their use. Meaning of course that in the right
situation one may be able to justify their own use of a nested comp.
However, for every day problems one wouldn't be encouraged to use them
so liberally as some tend to do.


So.. we have some sort of consensus then? This might be a rare
phenomenon on Usenet... :)

Cheers,

J.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying feedparser issues

[J Kenneth King]

John Nagle  writes:

> This really isn't the fault of the "feedparser" module, but it's
> worth mentioning.
>
> I have an application which needs to read each new item from a feed
> as it shows up, as efficiently as possible, because it's monitoring multiple
> feeds.  I want exactly one copy of each item as it comes in.
>
> In theory, this is easy.  Each time the feed is polled, pass in the
> timestamp and ID from the previous poll, and if nothing has changed,
> a 304 status should come back.
>
> Results are spotty.  It mostly works for Reuters.  It doesn't work
> for Twitter at all; Twitter updates the timestamp even when nothing changes.
> So items are routinely re-read.  (That has to be costing Twitter a huge
> amount of bandwidth from useless polls.)
>
> Some sites have changing feed etags because they're using multiple
> servers and a load balancer. These can be recognized because the same
> etags will show up again after a change.
>
> Items can supposedly be unduplicated by using the "etag" value.
> This almost works, but it's tricker than one might think.  On some feeds,
> an item might go away, yet come back in a later feed.  This happens with
> news feeds from major news sources, because they have priorities that
> don't show up in RSS.  High priority stories might push a low priority story
> off the feed, but it may come back later.  Also, every night at 00:00, some
> feeds like Reuters re-number everything.  The only thing that works reliably
> is comparing the story text.
>
>   John Nagle

I can't really offer much help, but I feel your pain.  I had to write
something of a similar system for a large company once and it hurt.
They mixed different formats with different protocols and man it was
something in the end.  The law of fuzzy inputs makes this stuff tough.

It may help to create a hash from the first x number of bytes of the
article text.  Then cache all the hashes in a local dbm-style
database.  We used berkely, but it doesn't really matter.  Whatever
way you can generate and store a keyed signature will allow you to do
a quick look up and see if you've already processed that article.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what I would like python.el to do (and maybe it does)

[J Kenneth King]

Giovanni Gherdovich  writes:

> Hello everybody,
>
> basically I'm writing here since I cannot
> make my python.el work (a major mode for writing
> python with emacs), but I would also like to share
> my user experience and tell you what I think
> an emacs mode should do, why do I like them
> and hopefully have some feedbacks to see if I
> misunderstood/underestimate something.
>
>
> == 1) my python.el doesn't behave very well ==
>
> I learnt somewhere
> http://www.emacswiki.org/emacs/PythonMode
> that there are two major emacs mode for python around:
> python-mode.el and python.el.
> Asking my Emacs 22.3.1 about the variable load-path
> issuing
> 'C-h v load-path RET
> I see that
> /usr/share/emacs/22.3/lisp/progmodes
> is in that path; there I find a file named
> python.elc
> which I assume to be some kind of emacs lisp bytecode
> since is pretty much unreadable.
>
> So I searched the web for a plain version of it,
> finding that the feature I use more, i.e.
> sending a line of a text file to the
> python buffer for evaluation (see below), correspond
> to the key sequence
>
> \C-c\C-c
>
> (well, it's python-send-buffer, so maybe not a single
> line but the whole buffer; the closest to my needs, anyway).
> However: I open my Emacs, issue M-x python-mode,
> then M-x run-python to have the interpreter in
> a second buffer, I type something in the
> first buffer and then C-c C-c, but nothing happens.
>
> Am I missing something?
> Do I have any hope of having some sort of
> send-line-to-python-buffer function working?
>
>
> == 2) How do I use emacs modes for interpreted languages ==
>
> Please note that what follows is just the personal perspective
> of an unexperienced user.
>
> Syntax highlighting is a great thing, but is not as critical
> to me as the feature I describe below.
>
> When I work with interpreted languages, I really hate doing it
> in the shell; after 20 commands I easily lose control on
> what happens and on which definitions are around.
>
> I use Emacs instead, so that I can have two buffers; in the
> first I type my expressions, in the second I evaluate them
> using some key bindings so that I can easily send the text
> from the first buffer to the second one line by line.
>
> In this way I can easily refactor my code, and eventually package it
> in a script if I like.
> Usually after a while the intepreter buffer is a big mess,
> so I restart it but my code is safe and sound in the first buffer.
>
> To do so, I don't really need a major mode, I admit; I just need
> to put the following code in my .emacs:
>
> (fset 'send-line-other-window
>  [?\C-a ?\C- ?\C-e ?M-w right
>   ?C-x ?o ?C-y return ?\C-x ?o])
> (global-set-key [f11] 'send-line-other-window)
>
> Then I open emacs, C-x 2 to have a second buffer,
> C-x o to switch to it and M-x shell to run bash in it.
> Then, in the case of python, I run "python" in the
> bash buffer. Then I type my code in the first and with F11
> I send lines to the interpreter.
>
> But since i like to do it The Right Way, I would
> like to let the python-mode worry about this...
>
> Sorry if this is just a bunch of obvious thoughts to most of you.
>
> Regards,
> Giovanni

I find that it does work, but unlike SLIME for lisp, it just imports the 
statement.

It confused me at first, but basically the interpreter doesn't provide
any feedback to emacs.

Try opening a python source file (start python-mode if you don't have
an autoload hook) and do C-c C-z to bring up the Python
interpreter. Type in a simple assignment statement (like "a = 1 + 2"
without the quotes) into the source file. Then just C-c C-c as
usual. I never get any feedback. Just C-x o to the interpreter and
print out the variable you just defined. It should be there.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What text editor is everyone using for Python

[J Kenneth King]

LittleGrasshopper  writes:

> With so many choices, I was wondering what editor is the one you
> prefer when coding Python, and why. I normally use vi, and just got
> into Python, so I am looking for suitable syntax files for it, and
> extra utilities. I dabbled with emacs at some point, but couldn't get
> through the key bindings for commands. I've never tried emacs with vi
> keybindings (I forgot the name of it) but I've been tempted.
>
> So what do you guys use, and why? Hopefully we can keep this civil.

Google should provide you with millions of responses.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What text editor is everyone using for Python

[J Kenneth King]

Lacrima  writes:

> I am new to python.
> And now I am using trial version of Wing IDE.
> But nobody mentioned it as a favourite editor.
> So should I buy it when trial is expired or there are better choices?

That is a slightly better question.

Try some of the free alternatives.  I do happen to use emacs.  It took
me quite a lot of adjusting to get used to it after being a vim user
for almost ten years.  Doesn't cost anything to give one a shot and
see if it works for you.  The choice of editor is a personal one
(hence the reluctance to answer your original question).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: which database is suitable for small applications

[J Kenneth King]

Jean-Michel Pichavant  writes:

> Kalyan Chakravarthy wrote:
>> Hi All,
>>   can any one suggest me which database I can use for my
>> small application(to store user names ,passwords, very few other
>> data.. )
>> I am using Python, Google Apps and guide me how to connect to
>> database, I am very new to these technologies
>>
>> Please help me
>>
>> Thanks in advance
>>
>> -- 
>> Regards
>> Kalyan
>>
> If you are really new to these technologies and don't want to spend
> some time on it, just serialize on the disk your data structure
> containing your names, password and so on
> (http://docs.python.org/library/pickle.html).
> Depending on your application you may not be that concerned with
> security/reliability issues.
>
> Of course if you want to learn more about it, go on. I personally use
> postgreSQL with the pgdb python module. But this is the only one I
> ever used so I'll let someone else more familiar with all the database
> types respond to you.
>
> Jean-Michel

sqlite is also a pretty lite database. Sits in a single file and the
libraries ship with Python (> 2.6 I think? or maybe 2.5?).

Serialization is always a neat option. With a hash table DB like
berkeley or some such you can get a pretty speedy (albeit simple)
persistence layer.

Really depends on what your app needs.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what I would like python.el to do (and maybe it does)

[J Kenneth King]

Piet van Oostrum  writes:

>>>>>> J Kenneth King  (JKK) wrote:
>
>>JKK> I find that it does work, but unlike SLIME for lisp, it just imports the 
>>statement.
>
>>JKK> It confused me at first, but basically the interpreter doesn't provide
>>JKK> any feedback to emacs.
>
>>JKK> Try opening a python source file (start python-mode if you don't have
>>JKK> an autoload hook) and do C-c C-z to bring up the Python
>>JKK> interpreter. Type in a simple assignment statement (like "a = 1 + 2"
>>JKK> without the quotes) into the source file. Then just C-c C-c as
>>JKK> usual. I never get any feedback. Just C-x o to the interpreter and
>>JKK> print out the variable you just defined. It should be there.
>
> What kind of feedback do you expect?

Well, that's the thing -- type a statement into a python interpreter and
you just get a new prompt.

LISP has a REPL, so you get some sort of feedback printed.

However, some sort of visual cue on the emacs side would be nice. Either
just flash the block of code being sent or a minibuffer message would be
nice.

Look for some SLIME tutorial videos on youtube to see some great
interpreter <-> editor interaction.

The stock Python interpreter probably wouldn't cut it close to something
like SLIME in terms of features, but the iPython package might be a
start.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what I would like python.el to do (and maybe it does)

[J Kenneth King]

Piet van Oostrum  writes:

>>>>>> J Kenneth King  (JKK) wrote:
>
>>JKK> Well, that's the thing -- type a statement into a python interpreter and
>>JKK> you just get a new prompt.
>
>>JKK> LISP has a REPL, so you get some sort of feedback printed.
>
> iPython also has a REPL, but only when you enter the Python code
> manually in the iPython window.
>
>>JKK> However, some sort of visual cue on the emacs side would be nice. Either
>>JKK> just flash the block of code being sent or a minibuffer message would be
>>JKK> nice.
>
>>JKK> Look for some SLIME tutorial videos on youtube to see some great
>>JKK> interpreter <-> editor interaction.
>
> I have tried out SLIME with SBCL (just some simple code) but I didn't
> like the feedback. I got unnecessary compiler warnings, and it was
> difficult to find some useful information in it.
>
>>JKK> The stock Python interpreter probably wouldn't cut it close to something
>>JKK> like SLIME in terms of features, but the iPython package might be a
>>JKK> start.
>
> For now the iPython package for me has more options than I have had time
> to try out.
> On the other hand when you execute some code form a Python file (with
> C-c C-c or C-c |) you get this
>  ## working on region in file /tmp/python-26084kfr.py... in the *Python*
> buffer which is very uninformative. This is generated by python-mode,
> not by iPython. You do get any output printed in the code, however, as
> well as exceptions.
>
> I have made a change in my Python mode such that the  
> ## working on region in file /tmp/python-26084kfr.py...
> message will be replaced by the actual code executed, if that code is
> not too big (size configurable). And that looks nicer. 
> -- 
> Piet van Oostrum 
> URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
> Private email: p...@vanoostrum.org

If you have a patch file for that, I'd be interested in trying it out. :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Programming Challenges for beginners?

[J Kenneth King]

astral orange <457r0...@gmail.com> writes:

> Hi-
>
> I am reading the online tutorial along with a book I bought on Python.
> I would like to test out what I know so far by solving programming
> challenges. Similar to what O'Reilly Learning Perl has. I really
> enjoyed the challenges at the end of the chapter and it really help me
> test out if I was truly taking in the material like I should. Could I
> get some suggestions on resources? Is there anywhere where I can go
> online (for free or purchase) for programming problems? I am familiar
> with sites like Code Chef...etc...but at this stage that is not the
> right 'venue' for me. I mainly need challenges like the ones they have
> in Learning Perl.
>
> Thanks again for all the help,
> 457r0

http://www.pythonchallenge.com/

I find this one neat because it uses riddles rather than straight
math.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Go versus Brand X

[J Kenneth King]

a...@pythoncraft.com (Aahz) writes:

> Comparing Go to another computer language -- do you recognize it?
>
> http://www.cowlark.com/2009-11-15-go/

If you skip to the conclusion, you'll be better off.

The author has an interesting point.

Go (the language) is not really ground-breaking.

I don't understand why they're so busy creating their own walled
garden...

Their own browser, their own programming languages, their own file
systems, etc.

Weird.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How decoupled are the Python frameworks?

[J Kenneth King]

shocks  writes:

> Hi
>
> I'm getting back into Python after a long break.  I've been developing
> large enterprise apps solely with Adobe Flex (ActionScript) for the
> past couple years.  During that time I've used a number of 'MVC'
> frameworks to glue the bits together - among them Cairngorm, a
> modified implementation of Cairngorm using the Presentation Model
> pattern, PureMVC, Mate (an IOC container but with an MVC
> implementation) and Parsley (IOC but you have to roll-you-own MVC).
> During that time I've been in large teams (30 Flex + 30 Java) to small
> teams (2 Flex + 1 Java).  The motivation of these frameworks is the
> decouple your concerns, allowing your apps to be more scalable, easier
> to test, and  supposedly easier to maintain.  Some do the decoupling
> better job than others, but there is also the question of "how
> decoupled is your code from the framework"?  It's all well and good
> having something clever working behind the scenes wiring and routing
> everything together, but I wonder where this leaves the code base if
> the framework, which was selected at the beginning of the project, is
> replaced with something else months or years later (i.e. the framework
> just doesn't scale as expected, the community involvement dies and
> it's no longer maintained properly, etc).  I've seen it happen and
> I've been experienced the pain of detangling massive amounts of code
> which is full of framework specific imports, methods and boilerplate
> code.  And then there's updating the unit tests!
>
> My question is how good are the current crop of Python frameworks?
> I've used Django twice in production and didn't like that much.  The
> implementation is Django specific for starters.  I've picked up Pylons
> and I'm trying that out.  I'm not sure how well it fares?  I do feel a
> bit uneasy about the code generation that some of the Python
> frameworks do.  Pylons creates something like 20 files for a
> 'helloworld'.  It does do some great things out of the box, but I
> wonder where that leaves your own code.  After spending 3-6 months on
> your Pylons webapp, how easy is it to move to something else?  Maybe
> one of the Python IOC once they mature.  What are some good techniques
> people are using to future (framework) proof their apps?
>
> I'm interested to hear people experiences with the various frameworks
> and how decoupled their code is from them.  The best of the current
> Flex frameworks for me is Parsley.  The only noticeable Parlsey code
> is an '[Inject]' meta tag here and there and a couple import
> statements.  All the complicated object creation and messaging is done
> higher up the chain.
>
> Cheers,
> Ben

I've stayed away from the Java world at least professionally... but I
think I understand where you're getting.

I'm not a huge fan of web development.  Which is rather
counter-intuitive for me to say since it's been the bulk of my work
for the past four years.  It's because there's no easy way to get
around the warts.  Websites are one thing but to try and think of
these things as "applications" becomes an expensive illusion to
maintain.

The problem with thinking about these things as applications with an
interface, a controller, and a model is: statelessness!  Oh also
serialization.  Getting around these issues are pretty much the raison
d'etre for web frameworks.  Without them we end up with PHP.

As far as swappable Python web frameworks... NONE. AFAIK, they all
require some commitment to tying your application to them.  However,
there are a crop of frameworks that do minimize the pain of such a
decision: web.py and bobo are two that I've been working with (though
it sounds like cherrypy would be very good at separating dispatching
from application code).  You could of course write your stuff closer
to the "metal" so to speak... WSGI would be about as low as I go.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about 'remote objects'

[J Kenneth King]

"Frank Millman"  writes:

> Hi all
>
> I am writing a multi-user business/accounting application. It is getting 
> rather complex and I am looking at how to, not exactly simplify it, but find 
> a way to manage the complexity.
>
> I have realised that it is logically made up of a number of services -
> database service with connection to database
> workflow engine for business processes
> services manager to handle automated services, such as web services
> client manager to service logins from client workstations
> possibly others
>
> I have made a start by splitting some of these off into separate modules and 
> running them in their own threads.
>
> I am concerned about scalability if they are all running on the same 
> machine, so I am looking into how to enable these services to run on 
> separate servers if required.

Have you finished the application already?

At my job we're still serving just over 1M+ web requests (a month),
processing 15k+ uploads, and searching through over 5M+ database records
a day.  We're still running on 3 boxes.  You can get a lot out of your
machines before you have to think about the complex task of
scaling/distributing.


> My first thought was to look into Pyro. It seems quite nice. One concern I 
> had was that it creates a separate thread for each object made available by 
> the server. My database server creates separate objects for each instance of 
> a row read in from the database, and with multiple users running multiple 
> applications, with each one opening multiple tables, this could run into 
> hundreds, so I was not sure if that would work.

It probably will work.

Pyro is a very nice framework and one that I've built a few applications
on.  It has a lot of flexible design patterns available.  Just look in
the examples included with the distribution.

>
> Then I read that the multiprocessing module allows processes to be spread 
> across multiple servers. The documentation is not as clear as Pyro's, but it 
> looks as if it could do what I want. I assume it would use processes rather 
> than threads to make multiple objects available, but I don't know if there 
> is a practical limit.

There is a theoretical limit to all of the resources on a machine.
Threads don't live outside of that limit.  They just have a speedier
start-up time and are able to communicate with one another in a single
process.  It doesn't sound like that will buy you a whole lot in your
application.

You can spawn as many processes as you need.

>
> Then I thought that, instead of the database server exposing each object 
> remotely, I could create one 'proxy' object on the server through which all 
> clients would communicate, and it in turn would communicate with each 
> instance locally.
>
> That felt more managable, but then I thought - why bother with remote 
> objects at all? Why not just run a SocketServer on the database server, and 
> design a mini-protocol to allow clients to make requests and receive 
> results. This is a technology I am already comfortable with, as this is how 
> I handle client workstation logins. If I did go this route, I could apply 
> the same principle to all the services.

Because unless you wrote your own database or are using some arcane
relic, it should already have its own configurable socket interface?

>
> I don't have the experience to make an informed decision at this point, so I 
> thought I would see if there is any consensus on the best way to go from 
> here.

Finish building the application.

Do the benchmarks.  Profile.  Optimize.

Find the clear boundaries of each component.

Build an API along those boundaries.

Add a network layer in front of the boundaries.  Pyro is a good choice,
twisted is also good.  Roll your own if you think you can do better or
it would fit your projects' needs.

> Is there any particular benefit in using remote objects as opposed to 
> writing a SocketServer?

Abstraction.  Pyro is just an abstraction over an RPC mechanism.
Nothing special about it.  Twisted has libraries to do the same thing.
Writing your own socket-level code can be messy if you don't do it
right.

>
> Any advice will be much appreciated.
>
> Thanks
>
> Frank Millman

Best of luck.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Perl to Python conversion

[J Kenneth King]

martin.sch...@gmail.com (Martin Schöön) writes:

> First off: I am new here and this is my first post after
> lurking for quite some time.

Hi.

> Second off: I don't know much Python---yet.

It's not a very big language. If you have experience programming in
other languages, you can probably pick it up in a day or two.

> Problem: I have come across a small open source application
> that I find quite useful. It does have one major flaw though.
> Its output is in imperial units. Converting isn't a big deal
> for occasional use but if I start to use this stuff on a
> regular basis...
>
> So I down-loaded the source code and found this thing is written
> in Perl.
>
> Should I learn enough Perl to add the conversion? Probably
> but this may be a nice excuse to get my Python education
> going and if I do I might as well re-do the user interface.

Well you can always call it from Python via subprocess (which basically
wraps a shell and has fancy ways putting data in and extracting data
out).

> If I do re-write this thing in Python I might need to learn both
> Perl and Python...

You'll need to know one of them rather well and enough of the other to
get by.  It's probably easier to know more Perl than Python since Perl
is a lot more expressive than Python (in the TMTOWTDI sense).  Frankly I
learned Perl before Python and find it rather easy to go between the
two.  YMMV.

> Hence, are there any Perl to Python converters? So far I
> have only found bridgekeeper which really is (was?) consultancy.
> Apart from that I only find people recommending a manual re-write.

It depends where the two languages vary from one another.

If the script your translating uses basic types or even simple classes
and typical control structures and operations then translating from one
to the other is a simple matter of copy-pasting the code and translating
the syntax and small bits of grammar.

However, in areas where there are high variations; you'll probably want
to stay away from it.  Perl has a lot of freedom to manipulate
references and the programmer can modify the language to suit their
needs.  So just be careful of code that uses these features as they are
difficult to translate into Python.

> Any thoughts/recommendations?

Depends:

- If you needed it done yesterday to get some work done, wrap the Perl
  script in a subprocess and buy yourself some time to think it over.  
- If your purpose is to learn Python, then start from scratch.  Use the
  Perl as a guide if there are any maths or algorithms you are unsure
  about.
- If you're just hacking around to learn stuff, learn a little of both.
  It will make you smarter if it doesn't confuse the heck out of you and
  make you quit before you finish. ;)

>
> TIA,
>
> /Martin

HTH
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[J Kenneth King]

Steven D'Aprano  writes:

> On Fri, 11 Dec 2009 19:20:21 -0500, Steve Holden wrote:
>
>> Simon Forman wrote:
>> [...]
>>> As far as the OP rant goes, my $0.02:  bad programmers will write bad
>>> code in any language, with any tool or system or environment they're
>>> given.  If you want to avoid bad code there's (apparently) no
>>> substitute for smrt programmers who are familiar with the tools they're
>>> using, not just the syntax but the underlying conceptual models as
>>> well.
>>> 
>> Hear, hear!
>
> That's all very well, but some languages and techniques encourage the 
> programmer to write bad code.

That's just BS.

Bad code doesn't just write itself.  Programmers write bad code.  And
ignorance is not an excuse.

Just because a language allows a programmer to write sloppy code doesn't
put the language at fault for the bad code programmers write with it.
Any half-way decent programmer should be cognisant of when they're
writing bad code and when they're writing good code.  They should be
able to admit that they don't know enough about a language to be writing
programs for money in it.  They should be able to see anti-patterns and
areas of their code that should be re-factored or re-written.

The real underlying problem is the human characteristic that allows us
to let ourselves believe that we're better than everyone else or more
simply, better than we really are.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[J Kenneth King]

r0g  writes:

> J Kenneth King wrote:
>> Steven D'Aprano  writes:
>> 
>>> On Fri, 11 Dec 2009 19:20:21 -0500, Steve Holden wrote:
>>>
> 
>
>>>> Hear, hear!
>>> That's all very well, but some languages and techniques encourage the 
>>> programmer to write bad code.
>> 
>> That's just BS.
>> 
>> Bad code doesn't just write itself.  Programmers write bad code.  And
>> ignorance is not an excuse.
>> 
>> Just because a language allows a programmer to write sloppy code doesn't
>> put the language at fault for the bad code programmers write with it.
>
>
>
> Okay, as long as you realize the corollary of your argument is:
>
> It is impossible for a language to encourage programmers to write good
> code and promote good programming practices by design.
>
> I'm not sure that's entirely true either.
>
> I think python's "one way to do something" design philosophy goes some
> way toward that, as does Smalltalk's enforced message passing. I think
> PHP's superglobals and namespacing encourage bad practices (or used to
> back in the day), as do Basic's GOTO and Ecmascript's prototype
> overriding.

I think your corollary is slightly misleading.

It would be more apt to say, "Just because a language allows a
programmer to write good code doesn't mean that the language is
responsible for the good code programmers write with it."

It is the responsibility of the programmer to recognize the advantages
and flaws of their tools.  PHP doesn't encourage a programmer to be a
bad programmer because it lacks name-spaces or because BASIC has GOTO
statements.  A bad programmer will be a bad programmer because they
don't understand what makes these features distinct, useful, or
damaging.

The language doesn't encourage anything.  It's just a medium like oil
paints and canvas.  A painting can be good or bad despite the medium it
is constructed on.  The skill of the painter is what matters.

>
> Surely a language CAN be said to encourage kludges and sloppiness if it
> allows a good way and a bad way and makes the bad way much easier to
> implement or understand for noobs.
>
> Roger.

The programmer can be encouraged to use kludges and produce sloppy
code.  Whether by ignorance or inflated ego.  Languages with more choice
just give them more rope to hang themselves with.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[J Kenneth King]

Neil Cerutti  writes:

> On 2009-12-16, J Kenneth King  wrote:
>> The language doesn't encourage anything.  It's just a medium
>> like oil paints and canvas.  A painting can be good or bad
>> despite the medium it is constructed on.  The skill of the
>> painter is what matters.
>
> Technically, oil paints do encourage a certain kind of painting.
> They can be layered on top of old paint easily, and they dry
> slowly, allowing you to slowly "build up" a painting in layers,
> and create effects with texture. If you try doing thse things
> with watercolors, and you'll probably be discouraged.
>
> I think a programming language does encourage a certain kind of
> code. Good code in one language can be poor in another.

It's a weak analogy on my part, but I think I do understand what you
mean here.

In regards to my original point, I think I just came up with a clearer
way to express it:

A language is a thing.  It may have syntax and semantics that bias it
towards the conventions and philosophies of its designers.  But in the
end, a language by itself would have a hard time convincing a human
being to adopt bad practises.

I believe it's the fault of the programmer who adopts those poor
practises.  Surely their acceptance of GOTO statements and
prototype-overloading are signs of their own preferences and ignorance?
It suggests to me that they learnt enough of one language to get by and
stopped thinking critically as soon as they sat in front of their
keyboard.

Throw an idiot behind a Python interpreter and it won't teach them a
damn thing unless they're capable of learning it on their own.  No
matter how well you manage to hard code your conventions into the
language.  Bad code is written by bad programmers, not bad programming
languages.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[J Kenneth King]

Lie Ryan  writes:

> On 12/17/2009 3:17 PM, J Kenneth King wrote:
>> A language is a thing.  It may have syntax and semantics that bias it
>> towards the conventions and philosophies of its designers.  But in the
>> end, a language by itself would have a hard time convincing a human
>> being to adopt bad practises.
>
> Perhaps someone should make a research whether if you teach a language
> to kids, where one group is taught the language filtered from "bad
> words" and another group is taught all the languages' "bad words" on
> purpose. Will one group have more behavioral problems compared to the
> other?

I would be curious to know, but the test is likely impossible without
trespassing on ethical boundaries. ;)

I would hypothesize that you would not find an increase in behavioural
problems.

a) Without cultural context "bad words" have little meaning

b) Behavioural issues can be attributed to several factors such as
physiology, health, environment, etc.

c) This has nothing to do with programming languages.  A programmer that
lacks critical thinking is a bad programmer.  The language they use has
no bearing on such human facilities.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[J Kenneth King]

Steven D'Aprano  writes:

> On Mon, 21 Dec 2009 11:44:29 -0500, J Kenneth King wrote:
>
>> A programmer that
>> lacks critical thinking is a bad programmer.  The language they use has
>> no bearing on such human facilities.
>
> That's nonsense, and I can demonstrate it by reference to a single 
> programming language, namely Python.
>
> For many years, Python had no ternary if operator:
>
> result = x if condition else y 
>
> Instead the accepted, idiomatic Python way of writing this was to use 
> short-circuit booleans:
>
> result = condition and x or y
>
> However this idiom is buggy! If x is a false-value (say, 0) then result 
> gets set to y no matter what the value of condition.
>
> This buggy idiom survived many years of Python development, missed by 
> virtually everyone. Even coders of the calibre of Raymond Hettinger (who 
> neither lacks critical thinking nor is a bad programmer) have been bitten 
> by this:
>
> "The construct can be error-prone.  When an error occurs it can be
> invisible to the person who wrote it.  I got bitten in published code
> that had survived testing and code review: ..."
>
> http://mail.python.org/pipermail/python-dev/2005-September/056510.html
>
>
> This is a clear and obvious case where a language feature (in this case, 
> the lack of a feature) encouraged an otherwise excellent coder to make an 
> error. It was a very subtle error, which was not picked up by the author, 
> the tests, or the coder reviewer(s). Had Python been different (either by 
> including a ternary if statement, or by forcing and/or to return bools 
> only) then this bug never would have occurred.
>
> Of course awful programmers will be awful programmers in any language, 
> and excellent programmers will be excellent programmers in many languages.
>
> (I say "many" rather than any deliberately. There's a reason why nobody 
> uses languages like Brainf*ck, Whitespace, Ook or Intercal for real work.)
>
> But most coders are neither awful nor excellent. The language DOES make a 
> difference: the quality of a technician depends partly on the quality of 
> his tools, and programmers are no different.
>
> If you don't believe me, imagine writing code in a language without 
> functions or loops, so you have to use GOTO for everything.

All very true.

But did the lack of ternary encourage Raymond to become a bad
programmer?

That is what I believe the core of the argument is.  Sure the misfeature
was over-looked by Raymond, but it took him (and perhaps the help of
others) to recognize it and fix it. That's because he's human and the
language is inert. He is smart and obviously has the cognitive
capabilities to recognize that the language has to change in order to be
a better tool.

It would be a different story if he just assumed that the misfeature was
actually a feature and that it was a good thing. In such a case would
Python the language be at fault or the people who write programs with
it?

Good tools make all the difference in the world, I'm not arguing that.

Just that the tools don't use us; we use them.  Programming in Python
doesn't instantly make me a better programmer.  It can certainly make me
think of myself as a good programmer though... ;)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[J Kenneth King]

Steven D'Aprano  writes:

> On Wed, 23 Dec 2009 10:55:19 -0500, J Kenneth King wrote:
>
>> Steven D'Aprano  writes:
>> 
>>> On Mon, 21 Dec 2009 11:44:29 -0500, J Kenneth King wrote:
>>>
>>>> A programmer that
>>>> lacks critical thinking is a bad programmer.  The language they use
>>>> has no bearing on such human facilities.
>>>
>>> That's nonsense, and I can demonstrate it by reference to a single
>>> programming language, namely Python.
>>>
>>> For many years, Python had no ternary if operator:
>
> [...]
>
>> But did the lack of ternary encourage Raymond to become a bad
>> programmer?
>
> No, but Raymond started off in a position of being an excellent 
> programmer. A single buggy idiom lead him to be slightly-less excellent 
> than he otherwise would have been. How many buggy idioms would it take to 
> lead him to become a mediocre coder, if he was unable to change languages?
>
> Because Python is generally an excellent language, the harm done by one 
> or two misfeatures is minor. But less excellent languages encourage 
> coding styles, techniques and idioms that encourage the programmer to 
> write poor code: either complicated, baroque, unreadable code; or slow 
> inefficient code; or buggy code. To avoid starting a flame war, I will 
> avoid mentioning PHP. *cough*
>
> Sometimes you know what you need to do to write non-buggy code, but 
> because covering all the corners are just Too Damn Hard in a certain 
> language, you simply lower your expectations. Error checking is tedious 
> and hard to get right in some languages, like C and Pascal, and hence 
> even good programmers can miss some errors.
>
> Different languages encourage different mind-sets in the programmer: C 
> encourages the coder to think at the low level of pointers and addresses, 
> and primarily about machine efficiency; Java encourages the use of big 
> object hierarchies and design patterns (it's hard to write lightweight 
> code in Java, so everything turns into heavyweight code); Perl encourages 
> cleverness and code-golf (writing a program in as few lines or characters 
> as possible); Haskell and Lisp encourage a heavily abstract approach that 
> often requires an elite coder to follow; Forth encourages you to think 
> like Yoda.

If anyone continues to follow bad idioms without questioning their
usefulness from time to time, I'd question their ability as a
programmer.  Critical thinking is important.  Which is why good programs
can be written in PHP, Forth, Lisp, Perl, and anything else.  However,
if a programmer thinks the only language they will ever need to know is
BF, they have a serious screw loose. ;)

> [...]
>> Good tools make all the difference in the world, I'm not arguing that.
>
> You appear to be arguing against that.

Maybe you need to reconsider my arguments.

It takes a good programmer to recognize the values and trade-offs of the
tools they work with.

Ignorance is not an excuse to blame the language.  It's too easy to say,
"Well Perl sucks because it encourages you to be a bad programmer
because it has all these features that let you shoot yourself in the
foot."  In reality, lots of really great programs are written in Perl
all the time and some very smart people write them.  It just so happens
that in hands of the educated, those very features are useful in certain
cases.

Python doesn't "encourage" you to be a better programmer.  It just
enforces particular idioms and conventions in its design.  As long as
the ignorant programmer follows them they should be better off.  Yet if
they are ignorant, no amount of encouragement will get them to think
critically about Python and find bugs in it.  They will have to rely on
the community of developers to do that thinking for them.

>
>> Just that the tools don't use us; we use them.
>
> Nobody said that tools use us.

But it is being suggested that they influence our thinking.

Pretty smart thing for a language to be able to do.

>> Programming in Python
>> doesn't instantly make me a better programmer.
>
> No, not instantly, but I would argue that after many years of coding in 
> Python you will be a better programmer than after the same number of 
> years of coding in PHP or Basic.

And my argument is that the human element is what will determine who is
better.

There are good programmers who can program in PHP.  Some of the biggest
websites on the Internet are programmed in it.  And like any language
I'm sure it has a good number of inefficiencies and bad design decisions
that the programmers using it had to work around.  Yet despite it being
a poor language in your opinion, they built successful program

Re: nested structure with "internal references"

[J Kenneth King]

Hendrik van Rooyen  writes:

> On Friday, 25 September 2009 19:11:06 Torsten Mohr wrote:
>
>> I'd like to use a nested structure in memory that consists
>> of dict()s and list()s, list entries can be dict()s, other list()s,
>> dict entries can be list()s or other dict()s.
>>
>> The lists and dicts can also contain int, float, string, ...
>
> This sounds terribly convoluted.
> What are you trying to do?
>
>>
>> But i'd also like to have something like a "reference" to another
>> entry.
>
> Everything and its brother in Python is an object, and things like lists and 
> dicts reference objects automagically.
>
>> I'd like to refer to another entry and not copy that entry, i need to
>> know later that this is a reference to another entry, i need to find
>> also access that entry then.
>
> Depending on how I read this, it is either trivial or impossible:
>
> A name is bound to an object by assignment.
> An object can have many names bound to it.
> A name can, at different times, refer to different objects.
> An object does not know which names are bound to it, or in what sequence it 
> was done.
>
> So you can go from name to object, but not the other way around.
> You can use the same name in a loop to refer to different objects, and in 
> each 
> iteration, use the name to store a reference to the current object in a list 
> or dict.
>
>> Is something like this possible in Python?
>
> Not too sure what you are trying to do.
>
>> The references only need to refer to entries in this structure.
>> The lists may change at runtime (entries removed / added), so
>> storing the index may not help.
>
> You could start off with a list of lists of lists, to any level of nesting 
> that you want.   This will give you a structure like a tree with branches and 
> twigs and leaves, but I am not sure if this is what you want or need, or if a 
> flat structure like third normal form would suffice, or if you need a 
> relational database.
>
> - Hendrik

I'm not really sure what he wants either, but it sounds suspiciously
like a linked list.

In regards to the OP, though not a true linked-list, Python objects can
be built with classes that describe essentially the same functionality.
Just be careful about keeping references to large in-memory objects (see
weakref) and try to avoid circular references in your mappings.

Cheers
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Object Relational Mappers are evil (a meditation)

[J Kenneth King]

Aaron Watters  writes:

> On Oct 16, 10:35 am, mario ruggier  wrote:
>> On Oct 5, 4:25 pm, Aaron Watters  wrote:
>>
>> > Occasionally I fantasize about making a non-trivial change
>> > to one of these programs, but I strongly resist going further
>> > than that because the ORM meatgrinder makes it somewhere
>> > between extremely unpleasant and impossible to make any
>> > non-trivial changes to a non-trivial program, especially after
>> > it has been populated with data.
>>
>> Object-relational mappers are like putting lipstick on a 
>> pig:http://gizmoweblog.blogspot.com/2006/10/putting-lipstick-on-pig.html
>>
>> m ;-)
>
> Cute, but wrong.  Using ORMs is better than using "Object databases".
>
> In my case I use Python to un data created by java/hibernate.
> If I was using a java based "object database" I would be simply stuck.
> At least if you use an ORM you have a way to access the information
> without writing a program in the programming language that the
> ORM was defined in.  Anyway, thanks for all the great comments on
> this thread from all you Sarcopterygii and Haplorrhini out there.

Data persistence isn't a "one-size fits all" problem.  It really depends
on the needs of the system.  Object databases solve the problem of
storing complex object graphs, deeply nested data structures, and
serializing language-specific objects like continuations or
what-have-you (but I think that last one is yet unsolved).  We all know
what RDBMS' are good for.  Neither is perfect for solving every data
persistence situation.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What IDE has good git and python support?

[J Kenneth King]

Aweks  writes:

> what do you use?

emacs.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python + twisted = Raindrop (in part)

[J Kenneth King]

Terry Reedy  writes:

> Rrom:
> First look: inside Mozilla's Raindrop messaging platform
>
> http://arstechnica.com/open-source/news/2009/10/first-look-inside-mozillas-raindrop-messaging-platform.ars
>
> "The backend components that are responsible for retrieving and
> processing messages are coded in Python on top of the Twisted
> networking framework."
>
> Ok, front end is html/Javascript, DB is erlang-based
>
> tjr

There are already developers grumbling about using Twisted.

https://wiki.mozilla.org/Raindrop/BackEndRoadmap#untwist_me.3F

I checked out the source myself and looked at the roadmaps and bug lists
to see if there were any patches I could work on.  But it doesn't look
all that interesting unless you're into Javascript.

Python is mainly used to fetch data.  Scheduling the fetch is left up to
the user/OS.  Python is also used for some "extensions" which are
basically document transformation scripts.  The extensions are stored in
the CouchDB and are eval'd asynchronously on each document in the
CouchDB.  A fairly expensive process to be sure.

Anyway, it's still really early in development so I'm sure they could
use some help.  :)

PS: Anyone wanna take a stab at their twisted pro/con list?

Cheers
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substituting list comprehensions for map()

[J Kenneth King]

Steven D'Aprano  writes:

> On Sun, 01 Nov 2009 23:54:16 -0800, Jon P. wrote:
>
>> I'd like to do:
>> 
>> resultlist = operandlist1 + operandlist2
>> 
>> where for example
>> 
>> operandlist1=[1,2,3,4,5]
>> operandlist2=[5,4,3,2,1]
>> 
>> and resultlist will become [6,6,6,6,6].  Using map(), I can do:
>> 
>> map(lambda op1,op2: op1 + op2, operandlist1, operandlist2)
>
>
> If the two lists are very large, it would be faster to use this:
>
>
> from operator import add
> map(add, operandlist1, operandlist2)

This is the best solution so far.

>
>
>> Is there any reasonable way to do this via a list comprehension ?
>
> [x+y for (x, y) in zip(operandlist1, operandlist2)]
>
> If the lists are huge, you can save some temporary memory by replacing 
> zip with itertools.izip.

I understand the OP was asking for it, but list comprehensions aren't
the best solution in this case... it would just be line noise.

List comprehensions are good for one-off transformations where it would
only create a one-time method for map to use.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substituting list comprehensions for map()

[J Kenneth King]

Ben Finney  writes:

> J Kenneth King  writes:
>
>> Steven D'Aprano  writes:
>>
>> > from operator import add
>> > map(add, operandlist1, operandlist2)
>>
>> This is the best solution so far.
>
> Strange to say it's a solution, when it doesn't solve the stated
> problem: to replace use of ‘map()’ with a list comprehension.

Granted I admit this later in my post.  It's not a solution to the OPs
request, but it is the best solution to such a case.

>
>> I understand the OP was asking for it, but list comprehensions aren't
>> the best solution in this case... it would just be line noise.
>
> I disagree; a list comprehension is often clearer than use of ‘map()’
> with a lambda form, and I find it to be so in this case.

The use of map expresses a value and implies the procedure by which it
is obtained.

The list comprehension expresses the procedure by which the value is
obtained.

Both have uses and in some cases a list comprehension is definitely
preferrable to a map operation.

However in this case the procedure by which we derive the value is not
important or even interesting.  It is much more succinct to think of the
operation as a value and express it accordingly.  There's no need to
clutter the mind with extra name bindings and iteration keywords.  They
won't make our idea any more clear.

dot_product = map(mul, vec1, vec2)

vs

dot_product = [a * b for a, b in zip(vec1, vec2)]

It's very clear, at least to me, what a dot-product is in this case.
Adding in the loop construct and name bindings doesn't enhance my
understanding of what a dot-product is.  I don't need to see the loop
construct at all in this case.  A dot product is simply the
multiplication of each element in a vector sequence.  It's more succinct
to simply think of the value rather then expressing the procedure to
construct that value.

This isn't a hammer issue.  Not every problem is a nail.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: substituting list comprehensions for map()

[J Kenneth King]

Steven D'Aprano  writes:

> On Tue, 03 Nov 2009 10:22:28 -0500, J Kenneth King wrote:
>
>> However in this case the procedure by which we derive the value is not
>> important or even interesting.  It is much more succinct to think of the
>> operation as a value and express it accordingly.  There's no need to
>> clutter the mind with extra name bindings and iteration keywords.  They
>> won't make our idea any more clear.
>> 
>> dot_product = map(mul, vec1, vec2)
>> 
>> vs
>> 
>> dot_product = [a * b for a, b in zip(vec1, vec2)]
>> 
>> It's very clear, at least to me, what a dot-product is in this case.
>
> Except it's not.
>
> The dot product of two vectors returns a scalar, not another vector:
> http://en.wikipedia.org/wiki/Dot_product
>
> So what you want is:
>
> dot_product = sum(map(mul, vec1, vec2))

Derh. Thanks for the catch. My bad.

>> Adding in the loop construct and name bindings doesn't enhance my
>> understanding of what a dot-product is.  I don't need to see the loop
>> construct at all in this case.  A dot product is simply the
>> multiplication of each element in a vector sequence.
>
> What you need is to define a function dot-product, and not hijack the 
> name for a local value. Then the function's implementation is irrelevant 
> to you: it could use a list comp, or could use map, it could use a for-
> loop, a while loop, recursion, or black magic:
>
> scalar = dot_product(vec1, vec2)

Even better.

But now I'm afraid the example is running away from the point.

So to summarize:

1. Extra name bindings and loop keywords aren't always easier to read.

2. Expressing what we want rather than how we get it is much more clear.

and (third dirty argument added)

3. List comprehensions leak their name bindings to the surrounding
scope. :p

Have a nice day. :)
-- 
http://mail.python.org/mailman/listinfo/python-list