Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Nick Coghlan 
Ben Finney wrote:
> Which then raises the question “what part of the set does it get?”,
> which the function signature does nothing to answer. I'm proposing that
> a no-parameters ‘set.get’ is needlessly confusing to think about.

The fact that set.get() is just set.pop() without removing the result
from the set seems perfectly straightforward.

> Since the use case is so specific, I would expect the name to be
> specific too, to better match the use case.

The use case is no more specific than set.pop().

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Martin (gzlist) 
On 24/10/2009, Nick Coghlan  wrote:
> Ben Finney wrote:
>> Which then raises the question “what part of the set does it get?”,
>> which the function signature does nothing to answer. I'm proposing that
>> a no-parameters ‘set.get’ is needlessly confusing to think about.
>
> The fact that set.get() is just set.pop() without removing the result
> from the set seems perfectly straightforward.

There's a different proposed meaning for `set.get` that's been
discussed on python-dev before:



That one I've had cause for before and no clean and fast way of
writing, this one I've always just done the for/break thing.

Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] First shot at some_set.get()

2009-10-24 Thread Steven D'Aprano 
On Sat, 24 Oct 2009 05:04:33 pm Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
>
>  > I'm not being tongue-in-cheek or sarcastic. My question was
> serious -- > if there is a moratorium, is there any reason to bother
> submitting > patches for functional changes to built-ins?
>
> Yes.  Python is open source.  Private and public forks are possible
> and (at least in principle) encouraged where the core project decides
> that the proposed changes are inappropriate (or should be deferred,
> as here).  Nevertheless, isn't the core Python project the obvious
> common point of contact for sharing such ideas, even if there is a
> moratorium on the code base itself?

No.

It's not obvious to me that the CPython tracker is the right place for 
patches for implementations that aren't for CPython.

It's not even obvious that there would be a common point of contact for 
private and public forks, let alone that it would be CPython's tracker. 
There are, by my count, 14 active and defunct implementations of Python 
to date, apart from CPython itself. How many of them currently use the 
CPython tracker to share patches? If the answer is "None", why would 
you expect future implementations and forks to be any different?



-- 
Steven D'Aprano
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Willi Richert 
Hi,

I agree. But, here are the pros/cons collected from the recent list repsonses:

Pro:
 - more readable
 - newbies will encounter one of the fastest solution (.get()) before trying 
slower "first solutions" like (iter(set).next())

Cons:
 - no name consensus. get() getany() arbitrary() ?
 - BDFL moratorium, which I find very wise (get() is, however, no language 
extension, but std lib extension, which Guido did not moratorize, didn't he?)
 - other classes should then also be extended, like frozenset

Regards,
wr

PS: what is the correct verb form of moratorium?

Am Samstag, 24. Oktober 2009 00:49:38 schrieb Steven D'Aprano:
> On Sat, 24 Oct 2009 07:53:24 am Willi Richert wrote:
> > Hi,
> >
> > surprised about the performance of for/break provided by Vitor, I did
> > some more testing. It revealed that indeed  we can forget the get()
> > (which was implemented as a stripped down pop()):
> 
> I don't understand that conclusion. According to your tests, your
> implementation of get() is as fast as "for x in set: break", and it's
> certainly much, much more readable and straightforward.
> 
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Steven D'Aprano 
On Sat, 24 Oct 2009 08:12:26 pm Martin (gzlist) wrote:

> On 24/10/2009, Nick Coghlan  wrote:
> > Ben Finney wrote:
> >> Which then raises the question “what part of the set does it
> >> get?”, which the function signature does nothing to answer. I'm
> >> proposing that a no-parameters ‘set.get’ is needlessly confusing
> >> to think about.
> >
> > The fact that set.get() is just set.pop() without removing the
> > result from the set seems perfectly straightforward.
>
> There's a different proposed meaning for `set.get` that's been
> discussed on python-dev before:
>
> 

That thread has an example of a use-case for extracting an item from a 
set, given the item itself: interning.

The current Python idiom for interning is to use a dict:

# store the canonical version
_cache[item] = item

# retrieve the interned version
item = _cache[item]

It has been argued that such interning is better done by sets rather 
than dicts, since this will save a pointer per item (plus any 
additional overhead dicts may have over that of sets). If this argument 
is accepted, then we want a fast, efficient operation to perform this:

def get(self, item):
"""Return an element of the set equal to item.
Useful for retrieving the canonical version of an element 
and for interning.
"""
for x in self:
if x == item:
return x
raise ValueError('item not in set')


The above is O(N); ideally it should be O(1).

Normally we don't care about identity, only equality, so if x and item 
above are equal we are indifferent about which one we use. But 
interning is a real example of a use-case where we do care about 
identity. Arguably it is inefficient and wasteful to use a dict for 
interning when sets could be just as fast while using less memory.

The other proposed semantics for set.get() are to retrieve an arbitrary 
element. It must be arbitrary, because elements in a set are unordered 
and unkeyed. This gives us the semantics of pop() without removing the 
element:

def get(self):
"""Return an arbitrary element of the set without removing it."""
for x in self:
return x
raise ValueError("empty set")

This is a frequent request, but I still don't know what the use-case is.

If you'll excuse me stating the obvious, both of these can easily be 
written as helper-functions. The main arguments for making them methods 
are:

(1) The first one is needlessly O(N) when it could be O(1).

(2) The second one is apparently a common need (although I personally 
have never needed it), forcing people to re-invent the wheel, sometimes 
incorrectly or inefficiently, e.g.:

def get(aset):
for element in aset:
pass
return element



-- 
Steven D'Aprano
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] First shot at some_set.get()

2009-10-24 Thread Martin v. Löwis 
> I'm not being tongue-in-cheek or sarcastic. My question was serious -- 
> if there is a moratorium, is there any reason to bother submitting 
> patches for functional changes to built-ins? A lot can change between 
> now and 2013, and I for one wouldn't bother making a patch that I knew 
> wouldn't even be considered for inclusion for four years, and would 
> likely need to be re-done even if it was accepted. Guido has said that 
> the purpose of the moratorium is to discourage changes to the language. 

I haven't been following the discussion, but I wouldn't expect that
a moratorium on language changes would rule out adding a method to the
set type.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Willi Richert 
Hi,

someone on this list mentioned that much of the s.get() time is spend on the 
name lookup for get(). That is indeed the case:

===
from timeit import *

stats = ["for i in xrange(1000): iter(s).next()   ",
 "for i in xrange(1000): \n\tfor x in s: \n\t\tbreak",
 "for i in xrange(1000): s.add(s.pop())   ",
 "for i in xrange(1000): s.get()  ",
 "g=s.get;\nfor i in xrange(1000): g()  "]

for stat in stats:
t = Timer(stat, setup="s=set(range(1000))")
print "Time for %s:\t %f"%(stat, t.timeit(number=1000))
==

Time for for i in xrange(1000): iter(s).next()   :   0.448227
Time for for i in xrange(1000):
for x in s:
break:   0.141669
Time for for i in xrange(1000): s.add(s.pop())   :   0.348055
Time for for i in xrange(1000): s.get()  :   0.148580
Time for g=s.get;
for i in xrange(1000): g()  :0.080563

So, now set.get() is indeed the fastest and preferable solution if you need 
massive amounts of retrieving elements from a set without removing them.

wr

Am Freitag, 23. Oktober 2009 22:53:24 schrieb Willi Richert:
> Hi,
> 
> surprised about the performance of for/break provided by Vitor, I did some
> more testing. It revealed that indeed  we can forget the get() (which was
> implemented as a stripped down pop()):
> 
> from timeit import *
> stats = ["for i in xrange(1000): iter(s).next()   ",
>  "for i in xrange(1000): \n\tfor x in s: \n\t\tbreak",
>  "for i in xrange(1000): s.add(s.pop())   ",
>  "for i in xrange(1000): s.get()  "]
> 
> for stat in stats:
> t = Timer(stat, setup="s=set(range(100))")
> try:
> print "Time for %s:\t %f"%(stat, t.timeit(number=1000))
> except:
> t.print_exc()
> 
> 
> 
> $ ./test_get.py
> Time for for i in xrange(1000): iter(s).next()   :   0.433080
> Time for for i in xrange(1000):
> for x in s:
> break:   0.148695
> Time for for i in xrange(1000): s.add(s.pop())   :   0.317418
> Time for for i in xrange(1000): s.get()  :   0.146673
> 
> 
> In some tests, for/break was even slightly faster then get().
> 
> As always, intuition regarding performance bottlenecks is flawed ;-)
> 
> Anyway, thanks for all the helpful comments, especially to Stefan for the
> http://comments.gmane.org/gmane.comp.python.ideas/5606 link.
> 
> Regards,
> wr
> 
> Am Freitag, 23. Oktober 2009 19:25:48 schrieb John Arbash Meinel:
> > Vitor Bosshard wrote:
> > > 2009/10/23 Willi Richert :
> > >> Hi,
> > >>
> > >> recently I wrote an algorithm, in which very often I had to get an
> > >> arbitrary element from a set without removing it.
> > >>
> > >> Three possibilities came to mind:
> > >>
> > >> 1.
> > >> x = some_set.pop()
> > >> some_set.add(x)
> > >>
> > >> 2.
> > >> for x in some_set:
> > >>break
> > >>
> > >> 3.
> > >> x = iter(some_set).next()
> > >>
> > >>
> > >> Of course, the third should be the fastest. It nevertheless goes
> > >> through all the iterator creation stuff, which costs some time. I
> > >> wondered, why the builtin set does not provide a more direct and
> > >> efficient way for retrieving some element without removing it. Is
> > >> there any reason for this?
> > >>
> > >> I imagine something like
> > >>
> > >> x = some_set.get()
> > >
> > > I see this as being useful for frozensets as well, where you can't get
> > > an arbitrary element easily due to the obvious lack of .pop(). I ran
> > > into this recently, when I had a frozenset that I knew had 1 element
> > > (it was the difference between 2 other sets), but couldn't get to that
> > > element easily (get the pun?)
> >
> > So in my testing (2) was actually the fastest. I assumed because .next()
> > was a function call overhead, while:
> > for x in some_set:
> >   break
> >
> > Was evaluated inline. It probably still has to call PyObject_GetIter,
> > however it doesn't have to create a stack frame for it.
> >
> > This is what "timeit" tells me. All runs are of the form:
> > python -m timeit -s "s = set([10])" ...
> >
> > 0.101us "for x in s: break; x"
> > 0.130us "for x in s: pass; x"
> > 0.234us -s "n = next; i = iter" "x = n(i(s)); x"
> > 0.248us "x = next(iter(s)); x"
> > 0.341us "x = iter(s).next(); x"
> >
> > So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x
> > faster than (iter(s).next()).
> > I was pretty surprised that it was 30% faster than "for x in s: pass". I
> > assume it has something to do with a potential "else:" statement?
> >
> > Note that all of these are significantly < 1us. So this only matters if
> > it is something you are doing often.
> >
> > I don't know your specific timings, but I would guess that:
> >   for x in s: break
> >
> > Is actually going to be faster than your
> >   s.get()
> >
> > Primarily because s.get() requires an attribute lookup. I would think
> > your version might be fas

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Alexander Belopolsky 
On Fri, Oct 23, 2009 at 6:46 PM, Steven D'Aprano  wrote:
> On Sat, 24 Oct 2009 06:04:12 am Terry Reedy wrote:
..
>> fwiw, I think the use case for this is sufficiently rare that it does
>> not need a separate method just for this purpose.
>
>
> And yet it keeps coming up, again and again... obviously people using
> sets in code think it has a use-case.
>
This reminds me a debate I had with Martin several years ago:

http://bugs.python.org/issue1507011

Here is the essence:

AB> I disagree with Martin. I think interning is a set
AB> operation and it is unfortunate that set API does not
AB> support it directly.

ML> I disagree with Alexander's last remark in several respects:
ML> set is indeed a container, and there is a way to get the
ML> elements back (namely, by enumerating them, or picking an
ML> arbitrary element for removal). There is no operation to check,
ML> on insertion of E, what the the prior element equal to E was
ML> (if any); there is only a check to determine whether E is in the
ML> set. The operation "give me the member equal but not identical
ML> to E" is conceptually a lookup operation; the mathematical set
ML> construct has no such operation, and the Python set models
ML> it closely. IOW, set is *not* a dict with key==value.

To me, however, a set seems to be a container that is a specialization
of a dict with values and keys being the same.   In this model, a
get() method, or even a __getitem__ with s[k] is k, is only natural.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] First shot at some_set.get()

2009-10-24 Thread geremy condra 
On Sat, Oct 24, 2009 at 11:28 AM, "Martin v. Löwis"  wrote:
>> I'm not being tongue-in-cheek or sarcastic. My question was serious --
>> if there is a moratorium, is there any reason to bother submitting
>> patches for functional changes to built-ins? A lot can change between
>> now and 2013, and I for one wouldn't bother making a patch that I knew
>> wouldn't even be considered for inclusion for four years, and would
>> likely need to be re-done even if it was accepted. Guido has said that
>> the purpose of the moratorium is to discourage changes to the language.
>
> I haven't been following the discussion, but I wouldn't expect that
> a moratorium on language changes would rule out adding a method to the
> set type.
>
> Regards,
> Martin

My understanding is that the moratorium would preclude changes to
the builtins. Is that not the case here?

Geremy Condra
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Guido van Rossum 
On Sat, Oct 24, 2009 at 10:34 AM, Alexander Belopolsky
 wrote:
> To me, however, a set seems to be a container that is a specialization
> of a dict with values and keys being the same.

That's absurd; the mapping provides nothing useful.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] First shot at some_set.get()

2009-10-24 Thread Guido van Rossum 
On Sat, Oct 24, 2009 at 10:55 AM, geremy condra  wrote:
> My understanding is that the moratorium would preclude changes to
> the builtins. Is that not the case here?

It is as yet undecided.

-- 
--Guido van Rossum

PS. My elbow needs a couple more weeks of rest. Limiting myself to
ultra-short emails.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Clean up Python/thread_*.h ?

2009-10-24 Thread Antoine Pitrou 

Hello,

I am wondering which of the files in Python/thread_*.h are really necessary
today. Looking at these files, I think most of them could perhaps be removed in
py3k. I've identified three categories of potentially removable files:

* The unused file: thread_wince.h

Windows CE has actually been using thread_nt.h since January 2009 (see
http://bugs.python.org/issue4893 ). thread_wince.h is still there, but it's
unused and grep brings no instance of it being mentioned anywhere in the source
tree.

* The (unsupported, untested?) files: thread_atheos.h, thread_cthread.h,
thread_lwp.h, thread_os2.h, thread_pth.h, thread_sgi.h.

These files refer to architectures which we probably have stopped caring about.
It is not even sure whether these files compile and work ok. Most of them have
not seen maintenance for years, except for the OS/2 file. In any case, it is
obvious they haven't received the level of attention and support that the
pthreads and NT versions have.
(also, according to http://www.lesstif.org/~amai/os2/html/porting.html, there is
a pthreads interface available for OS/2)

* The questionable files: thread_pth.h, thread_solaris.h.

GNU pth is probably obsolete on all platforms and superseced by the pthreads
API. According to http://www.gnu.org/software/pth/, the last pth version was
released in 2006 (and the changelog is rather unimpressive).
As for Solaris, according to the Sun website it should provide the pthreads API.
It was already documented in a 1999 manpage for SunOS 5.9:
http://docs.sun.com/app/docs/doc/816-0216/6m6ngupon?a=view .


Making a decision and removing the files considered unnecessary would clarify
what platforms still are/should be supported. Having less supported platforms
would also make improvements easier. Right now making an addition to the core
threading primitives is almost impossible since it would require to provide all
those platform-specific versions which almost nobody is competent for.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] interning

2009-10-24 Thread Antoine Pitrou 
Alexander Belopolsky  gmail.com> writes:
> 
> AB> I disagree with Martin. I think interning is a set
> AB> operation and it is unfortunate that set API does not
> AB> support it directly.
> 
> ML> I disagree with Alexander's last remark in several respects:
[...]
> ML> The operation "give me the member equal but not identical
> ML> to E" is conceptually a lookup operation; the mathematical set
> ML> construct has no such operation, and the Python set models
> ML> it closely. IOW, set is *not* a dict with key==value.

This looks like a debate of purity vs. practicality.

I don't have any opinion on a Python-facing API, but the interpreter's dict of
interned strings could probably converted to a set as a simple optimization.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Clean up Python/thread_*.h ?

2009-10-24 Thread Christian Heimes 
Antoine Pitrou wrote:
> * The unused file: thread_wince.h
> 
> Windows CE has actually been using thread_nt.h since January 2009 (see
> http://bugs.python.org/issue4893 ). thread_wince.h is still there, but it's
> unused and grep brings no instance of it being mentioned anywhere in the 
> source
> tree.

What about older versions of Windows CE? Maybe they still require the
wince thread header?

> 
> * The (unsupported, untested?) files: thread_atheos.h, thread_cthread.h,
> thread_lwp.h, thread_os2.h, thread_pth.h, thread_sgi.h.

thread_atheos.h
---

According to svn annotate the file was added on one revision and never
touched afterwards. It was added by MvL.

r27146 | loewis | 2002-06-11 08:22:31 +0200 (Tue, 11 Jun 2002) | 2 lines
Patch #488073: AtheOS port.

thread_cthread.h


Most lines were last modified or added in in revision 4018 (!), 12178,
16421 and 24967 by Guido, Thomas Wouters and MvL. The last change was:

r24967 | loewis | 2002-01-01 19:41:33 +0100 (Di, 01. Jan 2002) | 2 Zeilen
Patch #497098: build support for GNU/Hurd.

thread_lwp.h


The history of this file is almost identical to thread_cthread.h. No
changes since r23732 (2001-10-16)

thread_os2.h


The file seems to be maintained by Andrew Macintyre. The last change was
on r46919 (2006-06-13). We should ask him before we remove the file.

thread_pth.h


No changes since r15372 (2000-05-08), most of the files was last changed
by Guido and Thomas Wouters.

thread_sgi.h


Same as thread_cthread.h


I'mm +1 on removing the files. Let's ask Martin and Andrew before we
remove the wince and os2 thread files.

Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] updated PEP 389: argparse

2009-10-24 Thread Steven Bethard 
Sorry for the delay, but I've finally updated PEP 389, the argparse
PEP, based on all the feedback from python-dev. The full PEP is below,
but in short, the important changes are:

* The getopt module will *not* be deprecated.
* In Python 2, the -3 flag will cause deprecation warnings to be
issued for optparse.
* No casually visible deprecation warnings for optparse are expected
until Jun 2013.
* Support for {}-format strings will be added when one of the
converters is approved. This can be done at any point though, so it
shouldn't hold up inclusion of argparse.

Steve

--
PEP: 389
Title: argparse - new command line parsing module
Version: $Revision: 75674 $
Last-Modified: $Date: 2009-10-24 12:01:49 -0700 (Sat, 24 Oct 2009) $
Author: Steven Bethard 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 25-Sep-2009
Python-Version: 2.7 and 3.2
Post-History: 27-Sep-2009, 24-Oct-2009


Abstract

This PEP proposes inclusion of the argparse [1]_ module in the Python
standard library in Python 2.7 and 3.2.


Motivation
==
The argparse module is a command line parsing library which provides
more functionality than the existing command line parsing modules in
the standard library, getopt [2]_ and optparse [3]_. It includes
support for positional arguments (not just options), subcommands,
required options, options syntaxes like "/f" and "+rgb", zero-or-more
and one-or-more style arguments, and many other features the other
two lack.

The argparse module is also already a popular third-party replacement
for these modules. It is used in projects like IPython (the Scipy
Python shell) [4]_, is included in Debian testing and unstable [5]_,
and since 2007 has had various requests for its inclusion in the
standard library [6]_ [7]_ [8]_. This popularity suggests it may be
a valuable addition to the Python libraries.


Why aren't getopt and optparse enough?
==
One argument against adding argparse is that thare are "already two
different option parsing modules in the standard library" [9]_. The
following is a list of features provided by argparse but not present
in getopt or optparse:

* While it is true there are two *option* parsing libraries, there
  are no full command line parsing libraries -- both getopt and
  optparse support only options and have no support for positional
  arguments. The argparse module handles both, and as a result, is
  able to generate better help messages, avoiding redundancies like
  the ``usage=`` string usually required by optparse.

* The argparse module values practicality over purity. Thus, argparse
  allows required options and customization of which characters are
  used to identify options, while optparse explicitly states "the
  phrase 'required option' is self-contradictory" and that the option
  syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file``
  "are not supported by optparse, and they never will be".

* The argparse module allows options to accept a variable number of
  arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The
  optparse module provides an untested recipe for some part of this
  functionality [10]_ but admits that "things get hairy when you want
  an option to take a variable number of arguments."

* The argparse module supports subcommands, where a main command
  line parser dispatches to other command line parsers depending on
  the command line arguments. This is a common pattern in command
  line interfaces, e.g. ``svn co`` and ``svn up``.


Why isn't the functionality just being added to optparse?
=
Clearly all the above features offer improvements over what is
available through optparse. A reasonable question then is why these
features are not simply provided as patches to optparse, instead of
introducing an entirely new module. In fact, the original development
of argparse intended to do just that, but because of various fairly
constraining design decisions of optparse, this wasn't really
possible. Some of the problems included:

* The optparse module exposes the internals of its parsing algorithm.
  In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed
  to be available to callbacks [11]_. This makes it extremely
  difficult to improve the parsing algorithm as was necessary in
  argparse for proper handling of positional arguments and variable
  length arguments. For example, ``nargs='+'`` in argparse is matched
  using regular expressions and thus has no notion of things like
  ``parser.largs``.

* The optparse extension APIs are extremely complex. For example,
  just to use a simple custom string-to-object conversion function,
  you have to subclass ``Option``, hack class attributes, and then
  specify your custom option type to the parser, like this::

class MyOption(Option):
TYPES = Option.TYPES + ("mytype",)

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Terry Reedy 

Steven D'Aprano wrote:

On Sat, 24 Oct 2009 08:12:26 pm Martin (gzlist) wrote:



There's a different proposed meaning for `set.get` that's been
discussed on python-dev before:




For that case, I think the OP was mistaken mistaken to reject using a 
dict. He had objects with a key and wanted an index based on that key.


That thread has an example of a use-case for extracting an item from a 
set, given the item itself: interning.


The current Python idiom for interning is to use a dict:

# store the canonical version
_cache[item] = item

# retrieve the interned version
item = _cache[item]


This is an interesting use case as it turns on the difference between 
mathematics, where value is identity, and informatics, where the two 
concepts split. The above is meaningless in math, but *is* meaningful in 
informatics, where different objects can have the same set-membership 
value. For Python's builtin sets, set-membership 'value' is based on 
hash comparisons followed by equality comparison, rather than identity. 
The purpose of this is to make them more like mathematical sets.


But because of the value-identity distinction, they are not exactly like 
mathematical sets. Given that set()s implicitly map equivalence classes 
(which are not Python objects) to representitive members of that class 
(which are), then I agree and would even argue that there should be 
method of retrieving the one representative given any member.


(The above assumes that .__eq__ is properly defined for potential 
members so that they are properly partitioned into equivalence classes. 
This is not true when Decimals are mixed with other number classes.)


A counter-argument is that the implicit mapping is an implementation 
detail. A bit-mapped set class for a finite range of ints would not have 
this mapping. So an ABC for sets probably should not include such a method.


It has been argued that such interning is better done by sets rather 
than dicts, since this will save a pointer per item (plus any 
additional overhead dicts may have over that of sets). If this argument 
is accepted, then we want a fast, efficient operation to perform this:


def get(self, item):
"""Return an element of the set equal to item.
Useful for retrieving the canonical version of an element 
and for interning.

"""
for x in self:
if x == item:
return x
raise ValueError('item not in set')


The above is O(N); ideally it should be O(1).

Normally we don't care about identity, only equality, so if x and item 
above are equal we are indifferent about which one we use. But 
interning is a real example of a use-case where we do care about 
identity. Arguably it is inefficient and wasteful to use a dict for 
interning when sets could be just as fast while using less memory.


The other proposed semantics for set.get() are to retrieve an arbitrary 
element. It must be arbitrary, because elements in a set are unordered 
and unkeyed. This gives us the semantics of pop() without removing the 
element:


def get(self):
"""Return an arbitrary element of the set without removing it."""
for x in self:
return x
raise ValueError("empty set")

This is a frequent request, but I still don't know what the use-case is.

If you'll excuse me stating the obvious, both of these can easily be 
written as helper-functions. The main arguments for making them methods 
are:


(1) The first one is needlessly O(N) when it could be O(1).

(2) The second one is apparently a common need (although I personally 
have never needed it), forcing people to re-invent the wheel, sometimes 
incorrectly or inefficiently, e.g.:


def get(aset):
for element in aset:
pass
return element


Terry Jan Reedy


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Clean up Python/thread_*.h ?

2009-10-24 Thread Martin v. Löwis 
> Making a decision and removing the files considered unnecessary would clarify
> what platforms still are/should be supported.

I think any such removal should go through the PEP 11 process. Put a
#error into the files for 3.2, and a removal notice into the PEP, then
remove them in 3.3.

As for OS/2: this should probably stay.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Terry Reedy 

Guido van Rossum wrote:

On Sat, Oct 24, 2009 at 10:34 AM, Alexander Belopolsky
 wrote:

To me, however, a set seems to be a container that is a specialization
of a dict with values and keys being the same.


That's absurd; the mapping provides nothing useful.


Given Alexander's premise, I agree with your response. But his premise 
is wrong. Python's current builtin set class maps abstract equivalence 
classes to representative members. And this *is* useful. Mapping 
arbitrary members of such classes to representative members, sometimes 
called 'canonical', is a standard problem/goal in math. String interning 
is an application of this idea.


Terry Jan Reedy



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Terry Reedy 

Alexander Belopolsky wrote:

On Fri, Oct 23, 2009 at 6:46 PM, Steven D'Aprano  wrote:

On Sat, 24 Oct 2009 06:04:12 am Terry Reedy wrote:

..

fwiw, I think the use case for this is sufficiently rare that it does
not need a separate method just for this purpose.


And yet it keeps coming up, again and again... obviously people using
sets in code think it has a use-case.


This reminds me a debate I had with Martin several years ago:

http://bugs.python.org/issue1507011

Here is the essence:

AB> I disagree with Martin. I think interning is a set
AB> operation and it is unfortunate that set API does not
AB> support it directly.

ML> I disagree with Alexander's last remark in several respects:
ML> set is indeed a container, and there is a way to get the
ML> elements back (namely, by enumerating them, or picking an
ML> arbitrary element for removal). There is no operation to check,
ML> on insertion of E, what the the prior element equal to E was
ML> (if any); there is only a check to determine whether E is in the
ML> set. The operation "give me the member equal but not identical
ML> to E" is conceptually a lookup operation; the mathematical set
ML> construct has no such operation, and the Python set models
ML> it closely. IOW, set is *not* a dict with key==value.

To me, however, a set seems to be a container that is a specialization
of a dict with values and keys being the same.


As I explained in response to Steven, set()s *implicitly* map of 
abstract, non-object equivalence classes to a concrete, representative 
object/member of that class.


>  In this model, a

get() method, or even a __getitem__ with s[k] is k, is only natural.


No, if key and value were the same thing, the get method would be 
nonesense, as Guido said. But since they are not, since the implict key 
is an abstract class, retrieving the representative, perhaps canonical 
object given *any* member *is* natural. Being able to do so is also a 
standard practice in mathematics.


Terry Jan Reedy


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Clean up Python/thread_*.h ?

2009-10-24 Thread Antoine Pitrou 
Martin v. Löwis  v.loewis.de> writes:
> 
> > Making a decision and removing the files considered unnecessary would 
> > clarify
> > what platforms still are/should be supported.
> 
> I think any such removal should go through the PEP 11 process. Put a
> #error into the files for 3.2, and a removal notice into the PEP, then
> remove them in 3.3.

Ok, thanks! I see that AtheOS and BeOS are already marked unsupported in PEP 11.

What is your opinion about the Solaris-specific thread support? Is it still
necessary?

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Adam Olsen 
On Fri, Oct 23, 2009 at 11:04, Vitor Bosshard  wrote:
> I see this as being useful for frozensets as well, where you can't get
> an arbitrary element easily due to the obvious lack of .pop(). I ran
> into this recently, when I had a frozenset that I knew had 1 element
> (it was the difference between 2 other sets), but couldn't get to that
> element easily (get the pun?)

item, = set_of_one


-- 
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] updated PEP 389: argparse

2009-10-24 Thread Ben Finney 
Steven Bethard  writes:

> Discussion: sys.err and sys.exit
> 
> There were some concerns that argparse by default always writes to
> ``sys.err``
[…]

Unless, I'm missing something, this should replace ‘sys.err’ with
‘sys.stderr’ throughout.

-- 
 \ “Pinky, are you pondering what I'm pondering?” “I think so, |
  `\Brain, but why would anyone want to see Snow White and the |
_o__)   Seven Samurai?” —_Pinky and The Brain_ |
Ben Finney

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] updated PEP 389: argparse

2009-10-24 Thread Steven Bethard 
On Sat, Oct 24, 2009 at 4:34 PM, Ben Finney  wrote:
> Steven Bethard  writes:
>
>> Discussion: sys.err and sys.exit
>> 
>> There were some concerns that argparse by default always writes to
>> ``sys.err``
> […]
>
> Unless, I'm missing something, this should replace ‘sys.err’ with
> ‘sys.stderr’ throughout.

Yep, that's a typo. Thanks for the catch. It's fixed in SVN and the website:

http://www.python.org/dev/peps/pep-0389/

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
--- The Hiphopopotamus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread John Arbash Meinel 
Adam Olsen wrote:
> On Fri, Oct 23, 2009 at 11:04, Vitor Bosshard  wrote:
>> I see this as being useful for frozensets as well, where you can't get
>> an arbitrary element easily due to the obvious lack of .pop(). I ran
>> into this recently, when I had a frozenset that I knew had 1 element
>> (it was the difference between 2 other sets), but couldn't get to that
>> element easily (get the pun?)
> 
> item, = set_of_one
> 
> 

Interesting. It depends a bit on the speed of tuple unpacking, but
presumably that is quite fast. On my system it is pretty darn good:

0.101us "for x in s: break"
0.112us "x, = s"
0.122us "for x in s: pass"

So not quite as good as the for loop, but quite close.

John
=:->
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Alex Martelli 

Next(s) would seem good...

Alex

Sent from my iPhone

On Oct 24, 2009, at 6:47 PM, John Arbash Meinel > wrote:



Adam Olsen wrote:
On Fri, Oct 23, 2009 at 11:04, Vitor Bosshard   
wrote:
I see this as being useful for frozensets as well, where you can't  
get

an arbitrary element easily due to the obvious lack of .pop(). I ran
into this recently, when I had a frozenset that I knew had 1 element
(it was the difference between 2 other sets), but couldn't get to  
that

element easily (get the pun?)


item, = set_of_one




Interesting. It depends a bit on the speed of tuple unpacking, but
presumably that is quite fast. On my system it is pretty darn good:

0.101us "for x in s: break"
0.112us "x, = s"
0.122us "for x in s: pass"

So not quite as good as the for loop, but quite close.

John
=:->
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Nick Coghlan 
Terry Reedy wrote:
>>  In this model, a
>> get() method, or even a __getitem__ with s[k] is k, is only natural.
> 
> No, if key and value were the same thing, the get method would be
> nonesense, as Guido said. But since they are not, since the implict key
> is an abstract class, retrieving the representative, perhaps canonical
> object given *any* member *is* natural. Being able to do so is also a
> standard practice in mathematics.

To formalise this invariant to some degree: given a set "s" and two
items "k1" and "k2", such that "k1 in s" and "k2 in s" and "k1 == k2",
there is a single item "k" in s such that "k1 == k" and "k2 == k".

If __getitem__ were to be provided by sets, then the last part of that
invariant could be written "s[k1] is s[k2]".

If you actually care about that aspect of the semantics for a particular
application, it would be far clearer to use a full dict() and live with
the additional memory usage. While I can see how saving that
pointer-per-entry might make sense in some circumstances, for most I
would see it as a needlessly confusing micro-optimisation.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] updated PEP 389: argparse

2009-10-24 Thread Nick Coghlan 
Steven Bethard wrote:
> Sorry for the delay, but I've finally updated PEP 389, the argparse
> PEP, based on all the feedback from python-dev. The full PEP is below,
> but in short, the important changes are:
> 
> * The getopt module will *not* be deprecated.
> * In Python 2, the -3 flag will cause deprecation warnings to be
> issued for optparse.
> * No casually visible deprecation warnings for optparse are expected
> until Jun 2013.
> * Support for {}-format strings will be added when one of the
> converters is approved. This can be done at any point though, so it
> shouldn't hold up inclusion of argparse.

The new version provides a very good summary of and response to the
feedback on the previous version of the PEP.

+1 on adding the module :)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread Terry Reedy 

Alex Martelli wrote:

Next(s) would seem good...


That does not work. It has to be next(iter(s)), and that has been tried 
and eliminated because it is significantly slower.



Interesting. It depends a bit on the speed of tuple unpacking, but
presumably that is quite fast. On my system it is pretty darn good:

0.101us "for x in s: break"
0.112us "x, = s"


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com