Re: [Python-Dev] PEP 350: Codetags

[Phillip J. Eby]

At 03:35 PM 9/26/2005 -0700, Micah Elliott wrote:
>Please read/comment/vote.  This circulated as a pre-PEP proposal
>submitted to c.l.py on August 10, but has changed quite a bit since
>then.  I'm reposting this since it is now "Open (under consideration)"
>at .

This seems a little weird to me.  On the one hand, seems like a cool idea 
if you aren't using Eclipse or another IDE that tracks this stuff, but 
still need some kind of tracking system.  But, if that is the case, the 
notation seems a little bit overkill, especially with respect to tracking 
who's responsible - i.e., just you.

If you have a team that can agree to use the tools, I suppose it might be 
useful, but then I wonder, why not use something like Trac?

Finally, I think there should be something besides just a comment to 
distinguish these things; like starting with a symbol (e.g. # !FIXME), so 
that that documentation extraction tools can distinguish code tags from 
other documentation that just happens to start with a CAPITALIZED word.

Overall, I'm kind of -0.5.  It seems like a spec in search of an 
application.  The Motivation is sorely lacking - it reads like, "hey, it's 
optional and you can do stuff", where the stuff you can do is deferred to a 
later section, and is mostly stuff that could easily be done in other 
ways.  For example, FIT-style acceptance test documents, or Python doctest 
files go a long way towards documenting stories in association with tests, 
and they don't require you to cram things into comments.  (And because 
they're executable tests, they get kept up-to-date.)  Tracking bugfixes 
with code history is handled nicely by tools like Trac.  There are lots of 
Python documentation tools already.  And so on.  Really, it reads to me 
like you came up with the features to sell the format, instead of designing 
the format to implement specific features.

My suggestion: implement some tools, use them for a while, and come back 
with more focused use cases to show why only this format can work, and why 
the Python core developers should therefore use it.  I'm not saying that 
you can't have an informational PEP unless it should be used in the stdlib, 
mind you.  Just pointing out that if you can't convince the core developers 
it's useful, I'm thinking you'll have a hard time convincing the community 
at large to actually use it.  You need to actually have a better mousetrap 
to present before you ask people to move their cheese.  :)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python-Dev] PEP 350: Codetags

[Phillip J. Eby]

At 09:10 AM 9/28/2005 -0700, Micah Elliott wrote:
>I agree that proof of value is necessary.  Without a spec though it
>will be hard to get people to know about a convention/toolset, so it's
>a bit of a chicken-egg problem -- I can't have a pep until the tools are
>in use, but the tools won't be used until programmers have
>means/motivation to use them, a pep.

My point about the lack of motivation was that there was little reason 
shown why this should be a PEP instead of either:

1. Documentation for a specific tool, or group of tools
2. A specific project's process documentation

Are you proposing that this format be used by the Python developers for 
Python itself?  A process spec like this seems orthogonal to 
Python-the-language.

To put it another way, this seems like writing a PEP on how to do eXtreme 
Programming, or perhaps a PEP on how the blogging "trackback" protocol 
works.  Certainly you might implement those things using Python, but the 
spec itself seems entirely orthogonal to Python.  I don't really see why 
it's a PEP, as opposed to just a published spec on your own website, unless 
you intend for say, the Python stdlib to conform to it.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception raising, and performance implications.

[Phillip J. Eby]

leo wrote:
>
> You're absolutely right, in fact the code snippet from my OP was taken
> directly from inspect.currentframe. We're intending on using this in
> production, and I'm trying to gauge what the implications may be.

Use sys._getframe() instead; it doesn't raise an exception.


> Wow, I was unaware of this. If Python internally uses exceptions, maybe
> they aren't as detrimental as I thought.

Exceptions raised from C code and caught by C code use considerably
less resources, so a "for" loop catching a StopIteration raised by a
built-in iterator will have better performance than Python code
catching an exception raised in Python code.  So, it's not necessarily
the case that the "for" loop scenario matches your scenario(s).  Always
measure.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python's Performance

[Phillip J. Eby]

Laszlo Zsolt Nagy wrote:
> Dave wrote:
>
> > Hello All,
> >
> > I would like to gather some information on Python's runtime
> > performance. As far as I understand, it deals with a lot of string
> > objects. Does it require a lot string processing during program
> > execution? How does it handle such time-consuming operations? Is there
> > a way to find out how many string operations (perhaps in the
> > underlying system) ) it does during program execution?
>
> Do you want to know how many internal string operations are done inside
> the Python interpreter? I believe it is not a useful information. There
> are benchmarks testing the *real performance* of Python.
>
> For example: http://www.osnews.com/story.php?news_id=5602

Actually, that benchmark shows something rather interesting.  The C and
Java versions of the benchmark are much faster than Python on so-called
"64-bit arithmetic", but only Python computes the *correct* answer to
the benchmark!  The others overflow 64 bits at some point and lose
precision, resulting in a nonsense result that the author failed to
notice.

So, without meaning to, the benchmark author has demonstrated something
important about Python, which is that writing the obvious thing in
Python tends to work correctly, even if it sometimes takes longer to
run than it would take for another language to produce the wrong
answer.  :)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: setuptools, ez_setup over http proxy

[Phillip J. Eby]

yoda wrote:
> I've recently configured my network such that I use squid as a http
> proxy.  I'd now like to be able to use setuptools and ez_setup via this
> proxy.  Is this possible? If so, how do I do it?

The key thing is just that Python's urllib module needs to be able to
retrieve web pages and download files.  Try pulling up your Python
interpreter and retrieving a page with urllib.  If it works, you're
good to go, and no need to bother with anything else.

If it doesn't work, try Jay's suggestions for setting environment
variables, or consult the urllib doc and/or code for more details on
how urllib finds out what proxy to use.

> The most that the setuptools documentation says is
> (http://peak.telecommunity.com/DevCenter/setuptools):
>
> "If you are behind an NTLM-based firewall that prevents Python
> programs from accessing the net directly, you may wish to first install
> and use the APS proxy server, which lets you get past such firewalls in
> the same way that your web browser(s) do."

Yeah, I mention this because by far the most common cause of problems
for Windows users is that they're behind one of those infernal
NTLM-based firewalls and thus can't use urllib unaided.


> ps. I'm not sure that this is the right forum for this question.  If it
> isn't feel free to point me to the right place.

The distutils-sig mailing list is the best place to ensure you get an
answer relatively quickly, since I don't subscribe to this list, just
skim it from time to time.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: setuptools, ez_setup over http proxy

[Phillip J. Eby]

yoda wrote:
> I've recently configured my network such that I use squid as a http
> proxy.  I'd now like to be able to use setuptools and ez_setup via this
> proxy.  Is this possible? If so, how do I do it?

The key thing is just that Python's urllib module needs to be able to
retrieve web pages and download files.  Try pulling up your Python
interpreter and retrieving a page with urllib.  If it works, you're
good to go, and no need to bother with anything else.

If it doesn't work, try Jay's suggestions for setting environment
variables, or consult the urllib doc and/or code for more details on
how urllib finds out what proxy to use.

> The most that the setuptools documentation says is
> (http://peak.telecommunity.com/DevCenter/setuptools):
>
> "If you are behind an NTLM-based firewall that prevents Python
> programs from accessing the net directly, you may wish to first install
> and use the APS proxy server, which lets you get past such firewalls in
> the same way that your web browser(s) do."

Yeah, I mention this because by far the most common cause of problems
for Windows users is that they're behind one of those infernal
NTLM-based firewalls and thus can't use urllib unaided.


> ps. I'm not sure that this is the right forum for this question.  If it
> isn't feel free to point me to the right place.

The distutils-sig mailing list is the best place to ensure you get an
answer relatively quickly, since I don't subscribe to this list, just
skim it from time to time.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reinvent no more forever

[Phillip J. Eby]

Ben Finney wrote:
>   - Proliferation. What's the protocol when[1] someone else puts an
> (incompatible, differently-specified) Enum implementation into
> PyPI?

Either one of the two will be judged better, and the other will wither
away, or else each will be better for different circumstances, and
different people will use them.  Either way, not a problem.


>   - Naming. How many ways can a package providing an Enum be named?
> I'd prefer mine to be named "Enum" or "enum", but why should mine
> be the one that claims that name?

If you feel badly about it, just call it BFEnum, or SuperEnum, or make
up some other "branding" to use for your packages.  Otherwise, take the
rightful spoils of the pioneer by staking your claim to the name.  :)


>   - It's just a pretty simple type, with unit tests. Does this really
> justify a PyPI package?

Yes.  Excellent documentation would be a plus, of course, and perhaps
one of those screencasts that seem to be all the rage nowadays.  ;)


> I'd love to follow the mantra PJE espouses, but if it's good for one
> person it's probably good for countless others. How do we deal with
> that? What actions can we take in advance to prevent problems in
> future?

It's simple, really.  Ridicule and scorn are quite effective behavior
modification techniques for a community to employ in furthering its
operational goals.  So, when people step out of line, we'll just make
fun of them until they conform.  :)


> [1] Of course, someone already has. I prefer mine to theirs, hence the
> question.

Okay, so call yours "SuperEnum" or "PowerEnum" or "UltraEnum" or
"BetterEnum", "Enum-O-Matic", "Symbolitron"...

or just think about *why* yours is better, for *whom* it's better, and
*when*, and then give it a name that emphasizes one or more of those
things.  Even though "all the good domain names are taken", there still
seem to be an infinity of names remaining.  That's also true for PyPI.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reinvent no more forever

[Phillip J. Eby]

Dave Hansen wrote:
> On 17 Nov 2005 18:06:55 -0800 in comp.lang.python, "Phillip J. Eby"
> <[EMAIL PROTECTED]> wrote:
>
> [...]
> >
> >Okay, so call yours "SuperEnum" or "PowerEnum" or "UltraEnum" or
> >"BetterEnum", "Enum-O-Matic", "Symbolitron"...
>
> ITYM "EnumOMatic" or "Enum_O_Matic".  "Enum-O-Matic" is a syntax
> error.  Or worse...

Not in a PyPI project name, it isn't.  Project names aren't limited to
being Python identifiers.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Persist a class (not an instance)

[Phillip J. Eby]

David  Wahler wrote:
> Kent Johnson wrote:
> > Is there a way to persist a class definition (not a class instance,
> > the actual class) so it can be restored later? A naive approach
> > using pickle doesn't work:
> [snip]
> > The idea is to persist classes that are created and modified at runtime.
>
> I couldn't resist the challenge, so I decided to take a crack at it. My
> code is below. (I hope it's OK to post it even though it's a bit on the
> long side.) So far, it seems to work OK; the biggest caveat to be aware
> of is that functions' global context is not preserved.
>
> My approach was to use pickle's __reduce__ protocol to store functions
> and classes. Of course, you can't modify the built-in function and
> classobj types, so I subclassed Pickler to override them. The advantage
> of this is that you don't need an extension to the pickling data
> format, and you can use the standard unpickler. (The custom module
> still needs to have been imported, as it adds the classobj type to
> __builtins__.)
>
> Unfortunately, I'm not sure how to go about making it work for
> new-style classes. It would seem to involve messing with dictproxy and
> descriptor objects, and that's getting me into more unfamiliar
> territory.
>
> I'm sure there's a better way to do this; this seemed like "the
> simplest thing that could possibly work".

This is actually pretty sweet.  It seems to me that you'll be fine with
new-style classes if you just save dict(ob.__dict__) instead of trying
to save __dict__ directly, as that'll get rid of the dictproxy part.
There's no generic way to save descriptors, as far as I know, but you
can always register reducers for specific types, like property, and
user-defined descriptor classes are likely to be picklable anyway.

As for functions' global context, you could look to see if there's a
__name__ present, in which case you can save a reference to that
module's __dict__.  Otherwise, simply pickle the func_globals as-is.
Some folks might just want to do that anyway, if the code isn't
actually being loaded from a module.

Of course, the classic caveat regarding pickles and security applies to
all this.  That is, pickles and security don't mix.  If you want one,
you can't really get the other.  ;-)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python and generic programming

[Phillip J. Eby]

Ian Bicking <[EMAIL PROTECTED]> wrote in message news:<[EMAIL PROTECTED]>...
> Jive Dadson wrote:
> > If you have, for example, several data-types for matrices, how do you
> > write code that will automatically find the right routine for quickly
> > multiplying a vector by a diagonal matrix represented as a vector, and
> > automatically call the right code for multiplying by a sparse matrix
> > represented by some sparse coding, etc?
> 
> I haven't been following this thread, but Phillip Eby's recent 
> implementation of generic functions should be mentioned if it hasn't 
> been already:
> 
> http://dirtsimple.org/2004/11/generic-functions-have-landed.html
> 
> It might look something like this:
> 
> @dispatch.on('m1', 'm2')
> def matrix_mul(m1, m2):
>  "Multiply two matrices"
> 
> @matrix_mul.when(VectorMatrix, SparseMatrix)
> def matrix_mul_sparse(m1, m2):
>  "Special implementation"

Actually, it would be more like:

@dispatch.generic()
def matrix_mul(m1, m2):
"""Multiply two matrices"""

@matrix_mul.when("m1 in VectorMatrix and m2 in SparseMatrix")
def matrix_mul_sparse(m1, m2):
"""Special implementation"""

The 'dispatch.on()' decorator is for single-dispatch functions only. 
If you want to dispatch on multiple arguments or on logical
conditions, you use 'dispatch.generic()'.  The implementations for
single and multiple dispatch are completely separate, and use
different algorithms.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mixing metaclasses and exceptions

[Phillip J. Eby]

Jp Calderone wrote:
>   I'd skip that, though.  Your problem doesn't sound "Metaclass!" at
me.
> I wonder if you could elaborate on your usage?  Perhaps there's a
better
> solution which doesn't involve metaclasses at all.

I suspect he could *maybe* get by without the metaclass, but not
without custom descriptors, which don't always work for classic
classes.  In particular, if a Java exception has a static method, he'll
want to map that to a Python classmethod or staticmethod, and as far as
I recall, that stuff just doesn't work with classic classes.

In particular, if a Java class has both a static method and a
non-static method of the same name, there's no way that I know of to
map it into Python using a classic class; you *have* to have a
metaclass with a data descriptor in order to prevent a __dict__ lookup
on the class itself.

The only solution I can think of for this is to only use metaclasses if
you have the static/non-static naming conflicts, or other features of
the mapped class that require using a metaclass or other newstyle-only
features.  Or, alternatively, force all Throwable subclasses to be
implemented as classic classes.  :(

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: When Python *Eggs* better than Python *distutils*?? What's Eggs?

[Phillip J. Eby]

Paul Boddie wrote:
> Could anyone enlighten me/us as to why the Smart Package Manager [1]
> (written in Python, presented at EuroPython this year) isn't being more
> closely investigated as part of a suitable solution?

More closely investigated by whom, as a solution for what?  Surely
there is someone somewhere investigating it as a solution for
something, so your presupposition that it isn't would seem to imply
that you have some more specific person(s) and solution(s) in mind.  :)

If you are speaking with respect to setuptools, I would just point out
that it is intended to be packaging-system neutral.  While the
easy_install package manager will work on most platforms supported by
Python, it's intended only for tasks that cannot be accomplished using
the local package management system.  So, I've definitely "closely
investigated" package management tools "as part of a suitable
solution," and have recently added features to make eggs work better
with package management tools.  This consideration, however, does not
extend to providing any special integration with any *particular*
package management tools, especially since some systems (e.g. Windows)
have little or no such support, and a common usage environment for
Python web applications at least (shared hosting providers) offer no
way to *use* the system package manager at all.

That having been said, a few quick glances at Smart's source code show
me that Smart itself could perhaps benefit from eggs, for example to
handle plugins for channels and distribution types.  Its hook system,
for example, could be handled in a decentralized way.  Also, in theory
at least, it looks like Smart could be extended to recognize eggs and
manage them as well, but I haven't looked deeply into any of these
questions since they're not in scope for my efforts at this time.
However, it would certainly appear that *someone* could investigate
making Smart a bigger part of some "suitable solution" for something.
:)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: When Python *Eggs* better than Python *distutils*?? What's Eggs?

[Phillip J. Eby]

[EMAIL PROTECTED] wrote:
> I have been using distuils for a while and was wondering when
> Python Eggs (new project) is better?

If you have a relatively simple setup script, don't need to upload your
package to PyPI, and don't include any files other than .py files and C
extensions in your distribution, you won't benefit much from switching
to setuptools.

If you have a lot of non-code files, setuptools will let you get rid of
MANIFEST hassles and data file installation issues.  A fairly complete
list of setuptools features can be found here:

http://cheeseshop.python.org/pypi/setuptools

Most of these are just conveniences for the developer, that get rid of
a lot of the repetitious code you have to put in setup.py, especially
if you are doing anything that the distutils doesn't handle
automatically.

However, the "killer app" feature of setuptools is that it lets you
distribute a project that depends on other software available via PyPI.
 Instead of having to bundle the other project inside yours, or tell
users to manually download and install the dependencies.  If you use
setuptools, then you can take advantage of automatic dependency
download and installation, making it easier for you as a developer to
reuse existing open source Python code.  This is the real reason
setuptools and eggs and easy_install exist: to allow Python developers
to reuse code without bundling or manual dependency management --
regardless of platform.  (i.e., without them having to support rpm,
deb, msi, dpkg, pkgsrc, and all the other nine jillion packaging
systems out there).


> So basically Python Eggs precompiles and compresses
> binaries for you so you just have to load it to run
> your app?

Python eggs are a way of bundling additional information with a Python
project, that allows its dependencies to be checked and satisfied at
runtime, as well as allowing projects to provide plugins for other
projects.  There are several formats for this, but the '.egg' zipfile
format is a convenient one for *distributing* projects.  Whether you
keep them compressed or not when you install them is partly a matter of
whether the project is able to be used in compressed form, and whether
you want to be able to use certain documentation or debugging tools
that don't always work with zip files.

This .egg format is created by the "bdist_egg" command, which is
basically similar to bdist_wininst or bdist_rpm, in that it creates a
file you can then upload to PyPI or otherwise distribute.  The main
difference between .egg and win32.exe or .rpm is that .egg files can be
put directly on sys.path and used, while the other formats cannot.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python to use a non open source bug tracker?

[Phillip J. Eby]

Giovanni Bajo wrote:
> I am seriously concerned
> that the PSF infrastructure committee EVER considered non open-source
> applications for this. In fact, I thought that was an implicit requirement in
> the selection.

The goal of the selection process is to support the work of the Python
developers who have to actually *use* the bug tracking system, and
support its upkeep.  In accordance with the Zen of Python, Practicality
beats purity.

P.S. The Sourceforge tracker being used now and for the past several
years isn't open source, either, so it's hardly an emergency to have an
open source tracker *now*.

-- 
http://mail.python.org/mailman/listinfo/python-list