Re: [Python-Dev] Update to Python Documentation Website Request

2009-07-28 Thread David Lyon 
On Tue, 28 Jul 2009 07:55:11 +0200, "Martin v. Löwis" 
wrote:
> Yes, eggs have the same problem. That's one of the reasons they
> don't get integrated into Python.

Yes but egg_info is included in python...

and the egg is not

Hence, what goes in and what doesn't isn't always that rational. I'm
just accepting that for the moment.

David


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Implementing File Modes

2009-07-28 Thread Gregory P. Smith 
On Mon, Jul 27, 2009 at 5:32 PM, Glyph Lefkowitz wrote:
> On Mon, Jul 27, 2009 at 3:04 PM, Paul Moore  wrote:
>>
>> I like MRAB's idea of using a (non-standard) "e" flag to include
>> stderr. So "r" reads from stdout, "re" reads from stdout+stderr.
>>
>> Anything more complicated probably should just use "raw" Popen
>> objects. Don't overcomplicate the interface.
>
> In my opinion, mangling stderr and stdout together is already an
> overcomplication.  It shouldn't be implemented.
>
> It seems like a good idea, until you realize that subtle changes to your OS,
> environment, or buffering behavior may result in arbitrary, unparseable
> output.

Agreed.  Leave stderr support out of this.  People who need stderr
should use the full subprocess.Popen interface.  Quick hack unixy
types will just run their process using a shell (which already seems
to be the default based on the "ls -l" example) and add 2>&1.  This
functionality is basically the equivalent of adding the | symbol on
either or both ends of a filename given to open() in perl.  (but
without being so gross).

I do wonder why you're trying to make it behave exactly like open()
including the mode= syntax.

Why not just define several names based on the behavior?

 ProcessReadWrapper()
 ProcessWriteWrapper()
 ProcessReadWriteWrapper()

-gps

>
> For example, let's say you've got a program whose output is a list of lines,
> each one containing a number.  Sometimes it tries to import gtk, and fails
> to open its display.
>
> That's fine, and you can still deal with it, as long as the interleaved
> output looks like this:
>
> 100
> 200
> Gtk-WARNING **: cannot open display:
> 300
> 400
>
> but of course the output might (although unlikely with such small chunks of
> output) end up looking like this, instead:
>
> 100
> 2Gtk-WAR0NING0 **:
> can30not 0open display:
>
> 400
>
> this is the sort of thing which is much more likely to happen once you start
> dealing with large volumes of data, where there are more page-boundaries for
> your buffers to get confused around, and you are playing with buffering
> options to improve performance.  In other words, it's something that fails
> only at scale or under load, and is therefore extremely difficult to debug.
>
> This option might be okay if it were allowed only on subprocesses opened in
> a text mode, and if the buffering logic involved forced stderr and stdout to
> be line-delimited, and interleave only lines, rather than arbitrary chunks
> of bytes.  Of course then if you use this flag with a program that outputs
> binary data with no newlines it will buffer forever and crash your program
> with a MemoryError, but at least that's easy to debug when it happens.
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Update to Python Documentation Website Request

2009-07-28 Thread Paul Moore 
2009/7/28 David Lyon :
> On Tue, 28 Jul 2009 07:55:11 +0200, "Martin v. Löwis" 
> wrote:
>> Yes, eggs have the same problem. That's one of the reasons they
>> don't get integrated into Python.
>
> Yes but egg_info is included in python...
>
> and the egg is not
>
> Hence, what goes in and what doesn't isn't always that rational. I'm
> just accepting that for the moment.

egg_info data is in to allow "standard" (setup.py install and hence OS
package manager managed) packages to provide metadata in a
discoverable way. Using a format that is (reasonably) compatible with
setuptools is simply a matter of co-operating with existing de facto
standards.

Eggs themselves (as a distribution format) are just zip files with a
funny extension, and as such are supported by Python.

The infrastructure and philosophy around eggs (easy_install, the
various .pth file manipulations, multi-version installs, etc) are
supported by Python (in the trivial sense that they are possible) but
are not "blessed" by standard library inclusion, precisely because of
the issues being mentioned here.

Your package manager has the same issues as the egg infrastructure
(lack of integration with system package managers being the biggest
one) and so is not suitable for the standard library in precisely the
same way.

Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Py_TPFLAGS_HEAPTYPE too overloaded

2009-07-28 Thread Nick Coghlan 
Greg Ewing wrote:
> The only real concern would be if it were somehow possible to crash
> the interpreter by modifying the type dict. I don't see how that
> could happen -- but maybe someone else on python-dev knows more about
> this?

I believe a major part of the issue is that attempting to answer the
question "Will allowing mutating operation *X* on
object/type/float/int/etc create any interpreter stability or security
problems?" is awfully close to trying to prove a negative.

Certainly, at least some of the "generic" operations involving types
include additional sanity checks that are bypassed for the builtin types.

One specific example I can think of is that object.__hash__ is special
cased in a few places due to the way its definition interacts with the
definition of comparison operations. Allowing changes to the contents of
object's tp_hash slot could lead to much weirdness when it came to
__hash__ inheritance.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Py_TPFLAGS_HEAPTYPE too overloaded

2009-07-28 Thread Nick Coghlan 
Nick Coghlan wrote:
> One specific example I can think of is that object.__hash__ is special
> cased in a few places due to the way its definition interacts with the
> definition of comparison operations. Allowing changes to the contents of
> object's tp_hash slot could lead to much weirdness when it came to
> __hash__ inheritance.

Just thought of a much better example as I clicked send: the basic
numeric types (especially int) are locked down because they are
special-cased all over the place (including in the main interpreter eval
loop) in order to speed up simple arithmetic.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] License for code extracted from a Python library?

2009-07-28 Thread Rocky Bernstein 
Hi -

I'm not sure if this is the right place to ask this, but I don't know of a
more appropriate place.

Sometime around 2007 I first extracted a function that was in Python's Cmd
class and used that. Since then, I have modified and generalized it a bit
more and turned it into a Python egg called
columnize.
I also ported it to Ruby and made Ruby package or gem. A couple of
people/companies have asked about redistributing this and have asked for a
more MIT-like and less GPL-like license. I guess they will bundle this with
their proprietary code.

For the Python package, I used PSL Version 2, but is this legitimate?
code.google.com doesn't seem to recognize in their list of licenses so that
suggests what I did might not be the right thing. (So instead,
code.google.com I selected the MIT License.)

Can a MIT License be used for code extracted from Python's standard library?
Other comments or suggestions?

Thanks.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] License for code extracted from a Python library?

2009-07-28 Thread Nick Coghlan 
Rocky Bernstein wrote:
> Can a MIT License be used for code extracted from Python's standard
> library? Other comments or suggestions?

The extracted code itself would stay under the PSF license since you
don't have the rights to change the license on that. However, as the PSL
itself is a very permissive license, you can bundle it with code under
whatever other license you or users of your package choose to use (even
proprietary ones).

That said, while I know a fair bit about software licensing I'm not a
lawyer, and even if I was one, I still wouldn't be *your* lawyer, so
take what I have to say with a fairly large grain of salt! :)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Update to Python Documentation Website Request

2009-07-28 Thread David Lyon 
On Tue, 28 Jul 2009 11:50:00 +0100, Paul Moore  wrote:
> egg_info data is in to allow "standard" (setup.py install and hence OS
> package manager managed) packages to provide metadata in a
> discoverable way. Using a format that is (reasonably) compatible with
> setuptools is simply a matter of co-operating with existing de facto
> standards.
> 
> Eggs themselves (as a distribution format) are just zip files with a
> funny extension, and as such are supported by Python.

ok - I get it.

It's all one big monty python comedy thing where things are quirky
and inconsistent - there's little continuety from one scene to the
next but you have to stop sometimes and have a laugh... because
to word quote from you - "funny"

Your whole email whilst perphaps technically correct is terribly
difficult for a software engineering person to follow.

Monty python was only meant to be a funny film for entertainment,
not a philosophy for building software from

Why we can have the egg_info without allowing the egg is kind of
like having a chicken and egg story without allowing the chicken..
or is it the egg.. oh I lost track...

I think I need to watch the movie again to understand what's
happening here...

Let me go away confused... don't ask me any more questions or
elaborate with more answers. Thanks. :-)

David





___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Update to Python Documentation Website Request

2009-07-28 Thread Paul Moore 
2009/7/28 David Lyon :
> ok - I get it.
[...]
> Your whole email whilst perphaps technically correct is terribly
> difficult for a software engineering person to follow.

OK, I'm sorry if my attempts to help you didn't do so.

> Let me go away confused... don't ask me any more questions or
> elaborate with more answers. Thanks. :-)

Don't worry, I'll do my best not to from now on :-)

Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Update to Python Documentation Website Request

2009-07-28 Thread Nick Coghlan 
David Lyon wrote:
> Your whole email whilst perphaps technically correct is terribly
> difficult for a software engineering person to follow.

It made perfect sense to me.

The words "eggs" brings with it a whole lot more baggage than just the
sum of the technical parts in the language core that support them
(primarily distutils and zipimport). I find it unfortunate that the name
for the distutils metadata format contains the word "egg" because it
implies much greater consensus around the philosophy behind eggs than
actually exists.

A lot of the baggage associated with the "eggs" concept is related to
the inherent conflict between different approaches to dependency management:
1. Use the system package management system for everything (preferred by
many, perhaps even most, *nix sysadmins, but not an option on Windows)
2. Create a Python specific package management system independent of the
system package manager (an area dominated by setuptools, including both
eggs and non-zipped package distributions)
3. Bundle everything into a monolithic application or framework (the
typical approach on Windows with py2exe, but also the philosophy behind
tools like virtualenv)

All 3 of those philosophies have good arguments in their favour and they
all have good arguments against them as well. Your comments about your
package management system suggest that it is just yet another entrant in
category 2 and you have said nothing to allay the concerns of those that
despise that philosophy with a passion because of the problems it causes
them when trying to manage their systems using the first philosophy.

Since the Python constituency includes developers and system
administrators that favour all 3 philosophies (and probably other
variants that I haven't thought to describe), anything that makes it
into the standard library will need to adequately balance the interests
of all parties (e.g. as has occurred in the PEP 376 metadata enhancement
discussions).

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Update to Python Documentation Website Request

2009-07-28 Thread Terry Reedy 

Nick Coghlan wrote:

David Lyon wrote:

Your whole email whilst perphaps technically correct is terribly
difficult for a software engineering person to follow.


It made perfect sense to me.


Like David, I found it a bit disjointed too.


The words "eggs" brings with it a whole lot more baggage than just the
sum of the technical parts in the language core that support them
(primarily distutils and zipimport). I find it unfortunate that the name
for the distutils metadata format contains the word "egg" because it
implies much greater consensus around the philosophy behind eggs than
actually exists.

A lot of the baggage associated with the "eggs" concept is related to
the inherent conflict between different approaches to dependency management:
1. Use the system package management system for everything (preferred by
many, perhaps even most, *nix sysadmins, but not an option on Windows)
2. Create a Python specific package management system independent of the
system package manager (an area dominated by setuptools, including both
eggs and non-zipped package distributions)
3. Bundle everything into a monolithic application or framework (the
typical approach on Windows with py2exe, but also the philosophy behind
tools like virtualenv)

All 3 of those philosophies have good arguments in their favour and they
all have good arguments against them as well. Your comments about your
package management system suggest that it is just yet another entrant in
category 2 and you have said nothing to allay the concerns of those that
despise that philosophy with a passion because of the problems it causes
them when trying to manage their systems using the first philosophy.

Since the Python constituency includes developers and system
administrators that favour all 3 philosophies (and probably other
variants that I haven't thought to describe), anything that makes it
into the standard library will need to adequately balance the interests
of all parties (e.g. as has occurred in the PEP 376 metadata enhancement
discussions).


However, the above clarifies for me what the alternatives and issues 
are. Thank you for posting.


Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Update to Python Documentation Website Request

2009-07-28 Thread David Lyon 
On Wed, 29 Jul 2009 10:56:20 +1000, Nick Coghlan 
wrote:
> The words "eggs" brings with it a whole lot more baggage than just the
> sum of the technical parts in the language core that support them
> (primarily distutils and zipimport). 

Well, in this case, (talking metaphorically) we have the ability
to roll the eggs, crack a whole in them and suck out the contents
(deal with a package in a zipped egg)

So metaphorically we could say that python can work with egg shells..

As for what's in the egg... well lets just say that it's a bit floaty..

And perphaps that is the best way for things to be for a while.

> I find it unfortunate that the name
> for the distutils metadata format contains the word "egg" because it
> implies much greater consensus around the philosophy behind eggs than
> actually exists.

I see it a different way. I thinks eggs are simple - as in like a java
bean. A simple way to move a package from one place to another. 

What seems to have gone wrong is that there is a lot of pioneering
and interconnected and interdependent technology within them. They're
an egg.. but in the past they've had little monsters that bite your
fingers when you try to put them in the pan... :-)

I think "Egg" term is very good. But maybe we are best not trying
to over-specify their contents.

Maybe we should say it has certain things (EGG_INFO, PACKAGE DATA)
as in (YOLK, WHITE) and pretty much leave it at that. If more detail
is required - rtm.

> A lot of the baggage associated with the "eggs" concept is related to
> the inherent conflict between different approaches to dependency
> management:

That's not an egg problem. That's a packaging/metadata problem.

It's the package dependency issue that's the problem. That's a distutils
problem.

> 1. Use the system package management system for everything (preferred by
> many, perhaps even most, *nix sysadmins, but not an option on Windows)

Yes, because the package dependency issues are just so great.

> 2. Create a Python specific package management system independent of the
> system package manager (an area dominated by setuptools, including both
> eggs and non-zipped package distributions)

More work needs to go into distutils. Not taking away from any existing
work - but setuptools has relied on the deficiencies of distutils to
gain a foothold.

Fix distutils (give me time to think..) and the need for setuptools and
any further fork is probably negated.

> 3. Bundle everything into a monolithic application or framework (the
> typical approach on Windows with py2exe, but also the philosophy behind
> tools like virtualenv)

Windows users are using py2exe and other products over distutils. To a
normal windows developer, distutils makes imho no sense in the way that
it goes about things now.

For example, very simple concepts like "program directories", (where
you stick the program) isn't an available option in distutils. But
it is the first thing that you need to know in a windows program.

So it's very hard to get to step 1...

> Your comments about your
> package management system suggest that it is just yet another entrant in
> category 2 and you have said nothing to allay the concerns of those that
> despise that philosophy with a passion because of the problems it causes
> them when trying to manage their systems using the first philosophy.

I'm a windows user.. I can't be in category #2..

I'm in category #1, have nothing... (except now my own tool)

> Since the Python constituency includes developers and system
> administrators that favour all 3 philosophies (and probably other
> variants that I haven't thought to describe), anything that makes it
> into the standard library will need to adequately balance the interests
> of all parties (e.g. as has occurred in the PEP 376 metadata enhancement
> discussions).

Well at least you are saying that there is some way of reconciling
all the different options...

There's an awful lot to take in, and there must be 20,000 lines of
emails for every 1 line of python code that is required to fix this
thing.

Take care

David 

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com