Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

2014-04-19 Thread David Cournapeau
On Fri, Apr 18, 2014 at 11:28 PM, Donald Stufft  wrote:

>
> On Apr 18, 2014, at 6:24 PM, Nick Coghlan  wrote:
>
> > On 18 April 2014 18:17, Paul Moore  wrote:
> >> On 18 April 2014 22:57, Donald Stufft  wrote:
> >>> Maybe Nick meant ``pip install ipython[all]`` but I don’t actually
> know what that
> >>> includes. I’ve never used ipython except for the console.
> >>
> >> The hard bit is the QT Console, but that's because there aren't wheels
> >> for PySide AFAICT.
> >
> > IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
> > let you break programming out of the low level box of controlling the
> > computer, and connect it directly to the more universal high level
> > task of understanding and visualising the world.
> >
> > Regards,
> > Nick.
> >
> >>
> >> Paul
> >
> >
> >
> > --
> > Nick Coghlan   |   [email protected]   |   Brisbane, Australia
>
> FWIW It’s been David Cournapeau’s opinion (on Twitter at least) that
> some/all/most
> (I’m not sure exactly which) of these can be handled by Wheels (they just
> aren’t right now!).
>

Indeed, and the scipy community has been working on making wheels for new
releases. The details of the format does not matter as much as having one
format: at Enthought, we have been using the egg format for years to deploy
python, C/C++ libraries and other assets, but we would have been using
wheels if it existed at that time. Adding features like pre remove/post
install to wheels would be great, but that's a relatively simpler
discussion.

I agree with your sentiment that the main value of sumo distributions like
anaconda, active python or our own canopy is the binary packaging + making
sure it all works together. There will always be some limitations in making
those sumo distributions work seamlessly with 'standard' python, but those
are pretty much the same issues as e.g. linux integrators have.

If the python packaging efforts help the linux distribution integration, it
is very likely to help us too (us == sumo distributions builders) too.

David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] dict and required hashing

2014-04-19 Thread Armin Rigo
Hi Jim,

On 18 April 2014 23:46, Jim J. Jewett  wrote:
> (2)  Is "the item will be hashed at least once" a language guarantee?

I think that a reasonable implementation needs to hash at least once
all keys that are added to the dictionary.  Otherwise we end up, as
you said, with a dictionary that contains non-hashable keys: this
crashes when it grows and tries to convert from a list storage to a
hash table.  This is against the expectations and probably against the
language spec...

It's a bit unclear to me if the language spec requires "if [ ] in
mydict:" to raise TypeError, as the key doesn't get added to the dict
in this case.  Similarly for "[ ] in { }" where the dictionary is
empty anyway.  I would say yes, erring on the side of caution...


A bientôt,

Armin.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

2014-04-19 Thread Antoine Pitrou
On Fri, 18 Apr 2014 11:58:59 -0400
Nick Coghlan  wrote:
> 
> Software integrators:
> 
> * Linux distributions and other operating system vendors
> * Sumo redistributions (commercial or otherwise)
> * "Python based environments" (PTVS, Enthought Canopy, wakari.io,
> Python Anywhere, etc)
> * Software-as-a-Service developers
> * Device manufacturers
> * PC OEMs
> * creators of corporate "Standard Operating Environment" definitions
> * System integrators (IBM, Boeing et al)
> * Application developers (from simple CLI tools to OpenStack)

I don't understand this. Why are SaaS developpers or application
developpers (oh, why are they separate, by the way? :-)) "software
integrators" rather than "end users"? While Linux distributions, OS
vendors and the like will build their own Python from source, normal
developers will usually rely on an already existing packaging of Python
(an installer, or a distribution-provided package).

> For end users, Python is likely consumed as *part of something else*.

Well, even for some developers. Consider someone writing a Web
application with Django (*): are they interested in Django because of
Python, or in Python because of Django? How can you reliably make the
difference?

(*) or a video game with Ren'Py

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Antoine Pitrou
On Fri, 18 Apr 2014 22:31:29 -0400
Nick Coghlan  wrote:
> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.

I'm -1 on this. This is destroying the simplification effort of the
dict API in Python 3.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Giampaolo Rodola'
On Sat, Apr 19, 2014 at 4:31 AM, Nick Coghlan  wrote:

> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.
>

I don't see this as a key porting hassle *at all* and I don't understand
why they think this would significantly help their porting (it wouldn't).
The only real barrier is the str/bytes conversion, really, and this is even
more true for projects massively centered around IO, such as Twisted and,
I'm sure, the main (only?) reason why Twisted hasn't been ported yet. They
will get much more benefit from additions such as PEP-461, which is of
great help for verbose protocols such as FTP, not this.

-- 
Giampaolo - http://grodola.blogspot.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Kristján Valur Jónsson
Wouldn't "iterkeys" simply be an alias for "keys" and so on?
I'm +1 on that.
It is a signigificant portion of the incompatibility, and seems like such a 
minor concession to compatibility to make.
K

-Original Message-
From: Python-Dev [mailto:[email protected]] 
On Behalf Of Antoine Pitrou
Sent: 19. apríl 2014 09:36
To: [email protected]
Subject: Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

On Fri, 18 Apr 2014 22:31:29 -0400
Nick Coghlan  wrote:
> After spending some time talking to the folks at the PyCon Twisted 
> sprints, they persuaded me that adding back the iterkeys/values/items 
> methods for mapping objects would be a nice way to eliminate a key 
> porting hassle for them (and likely others), without significantly 
> increasing the complexity of Python 3.

I'm -1 on this. This is destroying the simplification effort of the dict API in 
Python 3.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Antoine Pitrou
On Sat, 19 Apr 2014 11:41:35 +
Kristján Valur Jónsson  wrote:
> Wouldn't "iterkeys" simply be an alias for "keys" and so on?

The PEP shows the following semantics:

def iterkeys(self):
return iter(self.keys())

def itervalues(self):
return iter(self.values())

def iteritems(self):
return iter(self.items())

Regards

Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Language Summit notes

2014-04-19 Thread Kristján Valur Jónsson


> -Original Message-
> From: Nick Coghlan [mailto:[email protected]]
> 
> > 2.   Feature enhancement to 2.8.  Take a robust and popular version of
> > python and add some of the language goodies that have been added to
> > 3.x and that don’t have an inherent 3.x aspect.  Yield from.  New exception
> model.
> > Stdlib enhancements such as futures.   The argument goes like this:  We
> have
> > a very popular platform out there with lots of momentum.  People want
> > incremental enhancements to it.  Why not give them what they want?
> > Bread and games and all that?  A Rockband cannot stay cooped up in a
> > studio producing experimental concept albums all the time.  That is death.
> > Sometimes it needs to go on tour and play old hits for the fans!
> 
> Do you know how much work a new Python 2.x release creates for people?
> All the redistributors have to update, books get outdated, a new wrinkle gets
> added to the compatibility matrix for everyone. A new Python 2.x release is
> simple untenable at this point in the transition
The key word here, "transition".  I’m not sure that everyone wants to transit.
This may be the core of the issue.   I know that this has been hashed to death 
before.  Never the less the pressure is there, I think and it would, I predict, 
be a crowd-pleaser.

> - it's a *massively* expensive way to achieve things that can be achieved
> more cheaply in other ways.
More cheaply from "our" point of view, perhaps.
> 
> Take yield from, for example. Hy is able to compile *LISP* syntax to Python
> AST structures. PEP 380 includes a semantic expansion of yield from in terms
> of yield. Is it *really* impossible to get "yield from"
> based code running in Python 2.6? Or have people just assumed it's not
> possible and never even tried, because the idea of using import hooks to
> backport syntax to earlier feature releases is too novel?

Some things are better done as language features than as complicated reverse 
hacks.
You keep saying that this and that can be done with add-on modules.  I think 
you underestimate the practical and psychological barrier towards those things. 
 Every new dependency on a package In some package directory is a new 
complication in a project.  It is something new you have t get, something 
requiing yet another virtualenv directory, something unblessed.
Another factor is the simply sheer size of Pypi by now.  How do you find 
things?  How do you even guess that things like "yield from" might be available 
as a package to pip install?  I know that this can be improved and tat there is 
work in process in doing that but the PyPi is still not core python and there 
is a gap that must be bridged for a user to start looking for solutions there.

> > 3.5 features
> >
> > When asked what should we aim for in 3.5, there were mostly some very
> > minor incremental changes suggested, IIRC.  In my opinion, the reason
> > 3.x has not caught on is that there is no real carrot there.  There is
> > no new vision, no killer feature.  Nothing that a programmer sees and
> > makes him say “Yeah! I want to program my next project using this feature,
> it will be super!”.
> 
> I *really* wish folks from North America, Europe and other regions where 8-
> bit encodings can handle their native language and where Anglicisation of
> terms to fit them into the ASCII identifier restriction poses no barrier to
> communication would stop trotting out this "no killer feature in Python 3"
> canard.
I intentionally didn't mention this because it is like the GIL.  It is a 
technical feature of the language, a refinement if you will.  But not a new 
thing in terms of language evolution.  Look, I have no disregard for the 
importance of this, myself coming from a non-ascii language.  I also worked on 
the internationalization of EVE, many years ago, and am familiar with the 
annoyance of implicit Unicode conversions.   I work in a company that uses 
unicode in all its products, new and old.  The other day a new stand alone 
service was developed.  I suggested to the developer that he might want to use 
Python 3 because it is unicode through and through.  He just shrugged, said it 
wasn't an issue.  He'll store the relevant database tables as unicode, get 
unicode out of the stuff, encode unicode in json and everything will just work. 
 While I'm not saying that the new model is not better (I think it is) it does 
come with some baggage, particularly in how it has been more cumbersome to work 
with bytes.  But anyway, this is why I didn"t mention unicode and why I don't 
count it as a killer feature.

 While it is *possible* to write internationalised and localised
> applications in it, Python 2's Unicode support is so broken that some people
> can't even run the interpreter from their home directory because it can't
> cope with their username. 
Years ago we implemented fixes to that for python 2.5.  core dev wasn't 
interested :).  

> If anyone is *ever* tempted to utter the words "

Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Steven D'Aprano
On Sat, Apr 19, 2014 at 11:41:35AM +, Kristján Valur Jónsson wrote:
> Wouldn't "iterkeys" simply be an alias for "keys" and so on?
> I'm +1 on that.

No.

[steve@ando ~]$ python2.7 -c "it = {}.iterkeys(); print it is iter(it)"
True
[steve@ando ~]$ python3.3 -c "it = {}.keys(); print(it is iter(it))"
False


> It is a signigificant portion of the incompatibility, and seems like 
> such a minor concession to compatibility to make.

I don't think it is a significant portion of incompatibility. Or at 
least, I think that the Twisted folks (or Nick, if he wants to speak for 
them) have to justify why it's significant.



-- 
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Nick Coghlan
On 19 Apr 2014 00:27, "Steven D'Aprano"  wrote:
>
> On Fri, Apr 18, 2014 at 10:31:29PM -0400, Nick Coghlan wrote:
> > After spending some time talking to the folks at the PyCon Twisted
> > sprints, they persuaded me that adding back the iterkeys/values/items
> > methods for mapping objects would be a nice way to eliminate a key
> > porting hassle for them (and likely others), without significantly
> > increasing the complexity of Python 3.
>
> It would also considerable add to the cruft of Python 3. One motive for
> going through the pain of Python 2 to 3 migration was to remove cruft.
> Adding it back in again is not just an aid to porting but actively
> making Python 3 a worse (well, "less better") experience.

The cruft is unavoidable in this case. The decision we face is *where the
cruft lives*, and *how much work is involved* in creating that cruft.

Status quo: we preserve the "purity" of the Python 3 mapping API, and
require every developer making the transition from Python 2 to Python 3
replace every occurrence of these methods with a helper function that is
not idiomatic code in either language.

My proposal: we add three trivial helper methods to the Python 3 mapping
API. For the cost of 3 additional methods that are easily explained in
terms of combining a builtin with other existing methods, a whole pile of
work just evaporates for affected segments of the community.

>
> So while I'm sympathetic to wanting to ease the 2/3 transition, even at
> the expense of re-introducing some cruft when unavoidable, I don't think
> that the difficulty of dealing with dict items|iteritems|viewitems etc.
> justifies re-adding cruft.

The cruft will be there regardless, the only question is whether it exists
in the definition of Python 3 or is distributed across all of the single
source projects supporting both 2 & 3.

> [...]
> > Rationale
> > =
> >
> > Similar in spirit to PEP 414 (which restored explicit Unicode literal
> > support in Python 3.3), this PEP is aimed primarily at helping users
> > that currently feel punished for making use of a feature that needed to
be
> > requested explicitly in Python 2, but was effectively made the default
> > behaviour in Python 3.
>
> "Feel punished"? That's awfully strong language. It may even be true, in
> the sense that some people *feel* that they are being punished, but I
> think the barrier to doing something about that needs to be a bit
> higher, namely that they *actually are* being punished.

Yes, they are *actually* being punished for using the memory efficient APIs
that were being added in Python 2.2. That's why the PEP covers how the
transition to Python 3 is harder for them than if they just hadn't cared
about memory efficiency in the first place.

They're Pythonistas too - they care about readability, and they *don't
like* having to add crufty helper functions to all their mapping
manipulation code. They also legitimately don't want to put up with all the
code churn that results from doing so, requiring additional reviews and
retesting of currently working code.

I spend a fair bit of time talking to users that have put a lot of work
into supporting a language transition that doesn't really help them
personally. A non-trivial number of them are deeply, viscerally angry about
what they see as pointlessly changing the spelling of a core language
feature for no real technical gain.

So let me be clear: this is *not* a proposal driven primarily by technical
considerations. Rather, it is a social one, where we do something simple
and easy and low cost for us to send a clear message to a set of users that
feel justifiably angry about the amount of work we imposed on other members
of the community by putting Python 2 into maintenance mode that we do value
their time and energy, and are willing to show some flexibility in adding
"not harmful" changes to Python 3 that don't significantly increase the
complexity of the language, while making it easier to write single source
code that looks like idiomatic Python code.

> I think that if "write a helper function" is punishment, then nearly
> every programmer is being punished *all the time*.

No, the punishment is "replace every usage of the memory efficient iterator
based APIs that have existed since Python 2.2 with a helper function
because we think that is a reasonable burden to place on you just so we can
avoid adding three trivial helper methods in Python 3".

Regards,
Nick.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Nick Coghlan
On 18 Apr 2014 23:08, "Benjamin Peterson"  wrote:
>
> On Fri, Apr 18, 2014, at 19:31, Nick Coghlan wrote:
> > After spending some time talking to the folks at the PyCon Twisted
> > sprints, they persuaded me that adding back the iterkeys/values/items
> > methods for mapping objects would be a nice way to eliminate a key
> > porting hassle for them (and likely others), without significantly
> > increasing the complexity of Python 3.
> >
> > I personally put this one in the same category as PEP 414 - not
> > particularly useful from a Python 3 perspective, but not really
> > harmful either, and helpful enough from a transition perspective to be
> > worth doing.
>
> It doesn't seem to be widely known that Python 2.7's dict has
> viewkeys()/viewvalues()/viewitems() methods which implement Python 3
> dictionary views. Thus, an alternate (or concurrent) proposal could be
> add these aliases to the Python 3 dictionary type. At any rate, the PEP
> should mention these methods' existence.

It does:
=
The view based APIs that were added to Python 2.7 don't actually help with
the transition process, as they don't exist in Python 3 and hence aren't
part of the common subset of Python 2 and Python 3, and also aren't
supported
by most Python 2 mappings (including the collection ABCs).
=

I should be more explicit that the other reason they don't really help is
because most potential single source code dates back further than 2.7, so
it's the iterator based APIs that are needed to avoid code churn when
migrating to single source.

Cheers,
Nick.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7

2014-04-19 Thread Antoine Pitrou
On Mon, 24 Mar 2014 10:10:18 +0100
"M.-A. Lemburg"  wrote:
> 
> The OpenSSL version used for 2.7.6 is 0.9.8y.
> 
> Upgrading to 1.0.0 or 1.0.1 will likely need a few minor tweaks, but
> not cause general breakage - at least that's my experience with
> the egenix-pyopenssl distribution.

For the record, if we had done that a few months ago, the breakage would
have been called Heartbleed.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Antoine Pitrou
On Sat, 19 Apr 2014 10:44:36 -0400
Nick Coghlan  wrote:
> 
> I should be more explicit that the other reason they don't really help is
> because most potential single source code dates back further than 2.7, so
> it's the iterator based APIs that are needed to avoid code churn when
> migrating to single source.

Why don't you just suggest adding convenience functions to six (or
another porting library)?

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Guido van Rossum
Does everyone involved know that "for x in d.iterkeys()" is equivalent to
"for x in d" and works the same in Python 2 and 3? Similarly, "list(d)" is
a simple, fast way to spell the Python 2 semantics of "d.keys()"  that
works in both versions (but I doubt it is much needed -- usually the actual
code follows up with sorting, so you should use sorted(d)).

This doesn't solve itervalues() and iteritems() but I expect those are less
common, and "for x, y in d.iteritems(): " is rewritten nicely as

  for x in d:
y = d[x]


If there is a measurable slowdown in the latter I would be totally okay
with some kind of one-element cache for the most recent lookup.

I get the social aspect of the PEP, but I think it's too high a price to
pay.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Chris McDonough

On 04/19/2014 07:41 AM, Kristján Valur Jónsson wrote:

Wouldn't "iterkeys" simply be an alias for "keys" and so on?
I'm +1 on that.
It is a signigificant portion of the incompatibility, and seems like such a 
minor concession to compatibility to make.
K


FWIW, I'm +1 on this and other minor changes and concessions to bw 
compat like this improve the subset language that 2/3 straddling 
projects need to use.


- C

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Nathaniel Smith
On Sat, Apr 19, 2014 at 3:31 AM, Nick Coghlan  wrote:
> Some Python 2 code that uses ``d.keys()`` may be migrated to Python 3
> (or the common subset of Python 2 and Python 3) without alteration, but
> *all* code using the iterator based API requires modification. Code that
> is migrating to the common subset of Python 2 and 3 and needs to retain the
> memory efficient implementation that avoids creating an unnecessary list
> object must switch away from using a method to instead using a helper
> function (such as those provided by the ``six`` module)

I don't know enough about the issues to have an opinion on the
proposal as a whole, but the foo.iterkeys() -> six.iterkeys(foo)
transformation strikes me as exactly the kind of change that can be
easily and accurately automated (as in modernize etc.). I assume Glyph
et al have considered this option -- do you know why it was rejected?

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Nick Coghlan
On 19 Apr 2014 10:53, "Guido van Rossum"  wrote:
>
> Does everyone involved know that "for x in d.iterkeys()" is equivalent to
"for x in d" and works the same in Python 2 and 3? Similarly, "list(d)" is
a simple, fast way to spell the Python 2 semantics of "d.keys()"  that
works in both versions (but I doubt it is much needed -- usually the actual
code follows up with sorting, so you should use sorted(d)).
>
> This doesn't solve itervalues() and iteritems() but I expect those are
less common, and "for x, y in d.iteritems(): " is rewritten nicely as
>
>   for x in d:
> y = d[x]
> 
>
> If there is a measurable slowdown in the latter I would be totally okay
with some kind of one-element cache for the most recent lookup.
>
> I get the social aspect of the PEP, but I think it's too high a price to
pay.

OK, I think the main thing I need to add to the PEP next is a clear
description of the current state of the art in translating these methods to
the common 2/3 subset across the following modes of interaction with the
existing Python 2 API:

* for loops
* comprehensions and generator expressions
* iterator object
* bound methods
* unbound methods

That may help clarify the tricky warts and edge cases that can arise when
moving from the current relatively straightforward and consistent method
based approach to a more complicated combination of dedicated syntax and
helper functions.

I also asked JP Calderone to trawl the Twisted code base for specific cases
where the status quo causes migration problems. Since my main argument is
that we should do this to save collective modification and review effort
for affected projects (i.e. I think it's a "death by 1000 cuts" situation
rather than a single gaping wound anyone can point to), it would be good if
those affected could help with quantifying the scale of the problem so we
can make a more informed trade-off between that work and future users of
Python 3 needing to learn about the existence of the proposed largely
redundant compatibility methods.

In several ways, I see my proposal as similar to what we did when PEP
8'ifying the threading.Thread API - the old camel case functions are still
there, but clearly marked as only existing for legacy compatibility reasons
(they may actually be completely undocumented in Py3 - I don't recall off
the top of my head). While I'm personally a big fan of cleaning up APIs and
strongly encouraging code modernisation, I've also become convinced that it
will be worth our while to start soliciting more quantitative feedback in
relation to various decisions to help make sure we're OK with the full
consequences of those design decisions.

Regards,
Nick.

>
> --
> --Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Ezio Melotti
Hi,

On Sat, Apr 19, 2014 at 4:14 PM, Steven D'Aprano  wrote:
> On Sat, Apr 19, 2014 at 11:41:35AM +, Kristján Valur Jónsson wrote:
>> It is a signigificant portion of the incompatibility, and seems like
>> such a minor concession to compatibility to make.
>
> I don't think it is a significant portion of incompatibility. Or at
> least, I think that the Twisted folks (or Nick, if he wants to speak for
> them) have to justify why it's significant.
>

Assuming this gets included in 3.5 (which will be released around the
end of 2015), are they planning to disregard all the previous 3.x
releases and then wait a couple more years (so 2018+) for 3.5 to
become common?
Are they going to support 3.3+ only (with u'...') and have extra cruft
for 3.3/3.4 to deal with the missing iter* methods and then remove the
cruft once 3.5 is the oldest 3.x release that is supported?
What happens if this addition will still not push people to move their
code to 3.x and similar requests are made for 3.6+ (and shift what I
just said for another 18 months)?

Best Regards,
Ezio Melotti

>
>
> --
> Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Stephen J. Turnbull
Guido van Rossum writes:

 > Does everyone involved know that "for x in d.iterkeys()" is
 > equivalent to "for x in d" and works the same in Python 2 and 3? 
[...]

 > This doesn't solve itervalues() and iteritems() but I expect those
 > are less common, and "for x, y in d.iteritems(): " is
 > rewritten nicely as
 >
 >   for x in d:
 > y = d[x]
 >

I suppose there's no way to get the compiler to both make "for x in d"
work as above, and make "for k, v in d" be equivalent to Python 2's
"for k, v in d.iteritems()"?  It seems totally analogous to getting
both "for x in list" and "for x, y in list_of_couples" to DTRT.  (To
me, anyway.)

You'd still be stuck on itervalues(), but at least you'd have the
option of "for _, v in d" (ie, the usual idiom for a value you're
going to ignore) without creating a list.




___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Benjamin Peterson


On Sat, Apr 19, 2014, at 9:30, Stephen J. Turnbull wrote:
> Guido van Rossum writes:
> 
>  > Does everyone involved know that "for x in d.iterkeys()" is
>  > equivalent to "for x in d" and works the same in Python 2 and 3? 
> [...]
> 
>  > This doesn't solve itervalues() and iteritems() but I expect those
>  > are less common, and "for x, y in d.iteritems(): " is
>  > rewritten nicely as
>  >
>  >   for x in d:
>  > y = d[x]
>  >
> 
> I suppose there's no way to get the compiler to both make "for x in d"
> work as above, and make "for k, v in d" be equivalent to Python 2's
> "for k, v in d.iteritems()"?  It seems totally analogous to getting
> both "for x in list" and "for x, y in list_of_couples" to DTRT.  (To
> me, anyway.)

That doesn't make sense. What if your keys are tuples?
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Guido van Rossum
Thinking about this more, I expect that an issue might be classes that
emulate dicts without being too formal about it (e.g. no ABCs) and then
porting code that works for both such class instances and dicts. Many
examples of such classes I've seen have a keys() method that returns a
list, and for various reasons these classes won't be rewritten to be more
like Python 3 as part of the port.

And then again, adding iterkeys() back to dict doesn't solve *that*
problem, and presumably such a class still has an __iter__() method, and
presumably the alternatives I proposed would still work.

I am also concerned about the dependency on Python 3.5 that we're building
here. I'd much rather be able to use Twisted sooner, with 3.3 or at least
3.4.


On Sat, Apr 19, 2014 at 9:17 AM, Nick Coghlan  wrote:

>
> On 19 Apr 2014 10:53, "Guido van Rossum"  wrote:
> >
> > Does everyone involved know that "for x in d.iterkeys()" is equivalent
> to "for x in d" and works the same in Python 2 and 3? Similarly, "list(d)"
> is a simple, fast way to spell the Python 2 semantics of "d.keys()"  that
> works in both versions (but I doubt it is much needed -- usually the actual
> code follows up with sorting, so you should use sorted(d)).
> >
> > This doesn't solve itervalues() and iteritems() but I expect those are
> less common, and "for x, y in d.iteritems(): " is rewritten nicely as
> >
> >   for x in d:
> > y = d[x]
> > 
> >
> > If there is a measurable slowdown in the latter I would be totally okay
> with some kind of one-element cache for the most recent lookup.
> >
> > I get the social aspect of the PEP, but I think it's too high a price to
> pay.
>
> OK, I think the main thing I need to add to the PEP next is a clear
> description of the current state of the art in translating these methods to
> the common 2/3 subset across the following modes of interaction with the
> existing Python 2 API:
>
> * for loops
> * comprehensions and generator expressions
> * iterator object
> * bound methods
> * unbound methods
>
> That may help clarify the tricky warts and edge cases that can arise when
> moving from the current relatively straightforward and consistent method
> based approach to a more complicated combination of dedicated syntax and
> helper functions.
>
> I also asked JP Calderone to trawl the Twisted code base for specific
> cases where the status quo causes migration problems. Since my main
> argument is that we should do this to save collective modification and
> review effort for affected projects (i.e. I think it's a "death by 1000
> cuts" situation rather than a single gaping wound anyone can point to), it
> would be good if those affected could help with quantifying the scale of
> the problem so we can make a more informed trade-off between that work and
> future users of Python 3 needing to learn about the existence of the
> proposed largely redundant compatibility methods.
>
> In several ways, I see my proposal as similar to what we did when PEP
> 8'ifying the threading.Thread API - the old camel case functions are still
> there, but clearly marked as only existing for legacy compatibility reasons
> (they may actually be completely undocumented in Py3 - I don't recall off
> the top of my head). While I'm personally a big fan of cleaning up APIs and
> strongly encouraging code modernisation, I've also become convinced that it
> will be worth our while to start soliciting more quantitative feedback in
> relation to various decisions to help make sure we're OK with the full
> consequences of those design decisions.
>
> Regards,
> Nick.
>
> >
> > --
> > --Guido van Rossum (python.org/~guido)
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Donald Stufft

On Apr 19, 2014, at 12:35 PM, Guido van Rossum  wrote:

> I am also concerned about the dependency on Python 3.5 that we're building 
> here. I'd much rather be able to use Twisted sooner, with 3.3 or at least 3.4.

Anyone who is planning on using the bytes modulo formatting is going to be 3.5+ 
anyways. It seems like trying to fit as many of these compatibility things as 
Python is willing to do into 3.5 is the best possible solution since it’s 
likely that for a lot of these hanger-ons 3.5 is likely to be a minimum target 
anyways.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Stephen J. Turnbull
Benjamin Peterson writes:

 > > I suppose there's no way to get the compiler to both make "for x in d"
 > > work as above, and make "for k, v in d" be equivalent to Python 2's
 > > "for k, v in d.iteritems()"?

 > That doesn't make sense. What if your keys are tuples?

Oh, I still think it makes sense.  Both x and k would be bound to the
key tuples.  For example it would "work" consistently with Common Lisp
multiple values.  And it's not clear to me that unpacking key tuples
would be used anywhere near as often as item unpacking.

But Python doesn't have other objects that behave like Common Lisp
multiple values, and it would change the meaning of currently correct
programs, so it's a non-starter.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] static typing of input arguments in signatures

2014-04-19 Thread Ethan Furman

On 04/13/2014 04:58 PM, R. David Murray wrote:

On Sun, 13 Apr 2014 15:59:36 -0400, Terry Reedy wrote:

On 4/13/2014 4:11 AM, �ukasz Langa wrote:

On Apr 13, 2014, at 12:48 AM, Stefan Behnel wrote:


So, what I've learned from seven years of Cython is that static typing in
signatures is actually less interesting than you might think at first
sight. It might be ok for documentation purposes, although its verboseness
makes that also a bit questionable.


You raise a valid point that type hinting a dict instead of a Mapping is
likely to create an overly limiting API. This sort of error is however
quite easy to fix forward.

To counter, we’ve had multiple data points during the summit suggesting
that duck typing is not really that often used in production code. In
other words, after initial prototyping and testing, an API of a callable
gets settled and is later used with a very limited number of types.


For public library code, where the use case is not known, apis should
usually be as generic as sensible. For private library code, I can
imagine that apis are and possibly even should be limited to classes
actually used. There naturally is a bit of a bias here for public code.


The way *I* heard it was that types of the input arguments did not, in
general, change *during the running of an application*.  That doesn't
mean that the types wouldn't be different in a different application,
which means that in a library that duck types, duck typing is indeed used
in production, it's just that the types don't *change* for a production
application.  So, yeah, pretty much what Terry said about library code
versus application code.  That is, after all, what duck typing is about,
and there is a *reason* we use it.


Personal experience:  I have my own copy of paramiko because it type checks for 
strings, and I routinely use a str-subclass.

--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Nick Coghlan
On 19 Apr 2014 12:29, "Ezio Melotti"  wrote:
>
> Hi,
>
> On Sat, Apr 19, 2014 at 4:14 PM, Steven D'Aprano 
wrote:
> > On Sat, Apr 19, 2014 at 11:41:35AM +, Kristján Valur Jónsson wrote:
> >> It is a signigificant portion of the incompatibility, and seems like
> >> such a minor concession to compatibility to make.
> >
> > I don't think it is a significant portion of incompatibility. Or at
> > least, I think that the Twisted folks (or Nick, if he wants to speak for
> > them) have to justify why it's significant.
> >
>
> Assuming this gets included in 3.5 (which will be released around the
> end of 2015), are they planning to disregard all the previous 3.x
> releases and then wait a couple more years (so 2018+) for 3.5 to
> become common?
> Are they going to support 3.3+ only (with u'...') and have extra cruft
> for 3.3/3.4 to deal with the missing iter* methods and then remove the
> cruft once 3.5 is the oldest 3.x release that is supported?
> What happens if this addition will still not push people to move their
> code to 3.x and similar requests are made for 3.6+ (and shift what I
> just said for another 18 months)?

This isn't really about porting major libraries or frameworks (except
perhaps Twisted, which was already blocked by the binary interpolation
issue anyway), but about chipping away at the barriers to migration for the
large custom in-house code bases, and the wide array of Python
*applications* that are part of the reason that people pay companies like
Red Hat to provide long term support.

Cheers,
Nick.

>
> Best Regards,
> Ezio Melotti
>
> >
> >
> > --
> > Steven
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Barry Warsaw
On Apr 19, 2014, at 02:12 PM, Giampaolo Rodola' wrote:

>I don't see this as a key porting hassle *at all* and I don't understand
>why they think this would significantly help their porting (it wouldn't).
>The only real barrier is the str/bytes conversion, really, and this is even
>more true for projects massively centered around IO, such as Twisted and,
>I'm sure, the main (only?) reason why Twisted hasn't been ported yet.

I agree.  I've been trying to get rid of iter*() when porting because most of
the time, there is no significant memory savings to be achieved anyway.

-Barry
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Eric Snow
On Sat, Apr 19, 2014 at 4:56 PM, Barry Warsaw  wrote:
> On Apr 19, 2014, at 02:12 PM, Giampaolo Rodola' wrote:
>
>>I don't see this as a key porting hassle *at all* and I don't understand
>>why they think this would significantly help their porting (it wouldn't).
>>The only real barrier is the str/bytes conversion, really, and this is even
>>more true for projects massively centered around IO, such as Twisted and,
>>I'm sure, the main (only?) reason why Twisted hasn't been ported yet.
>
> I agree.  I've been trying to get rid of iter*() when porting because most of
> the time, there is no significant memory savings to be achieved anyway.

+1

-eric
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Raymond Hettinger

On Apr 18, 2014, at 7:31 PM, Nick Coghlan  wrote:

> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.

I'm not keen on letting Python 2 leak into Python 3. 
That defeats one of the goals of Python 3 (simplification
and leaving legacy APIs behind a in fresh start).

As a Python instructor and coach, I can report that we
already have too many methods on dictionaries and
that it creates a usability obstacle when deciding which
methods to use.

In Python 2.7, a dir(dict) or help(dict) presents too many
ways to do it.   In Python 3.4, we finally have a clean
mapping API and it would be a pitty to clutter it up
in perpetuity.


Raymond




___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Greg Ewing

Stephen J. Turnbull wrote:

Benjamin Peterson writes:

 > > I suppose there's no way to get the compiler to both make "for x in d"
 > > work as above, and make "for k, v in d" be equivalent to Python 2's
 > > "for k, v in d.iteritems()"?

it would change the meaning of currently correct
programs, so it's a non-starter.


Maybe what's wanted is a function analogous to enumerate() for
mappings instead of sequences. Picking a semi-arbitrary name
for now:

   for k, v in tabulate(d):
  ...

It could be special-cased to recognise dicts and do the appropriate
thing for the Python version concerned. If it doesn't recognise the
type, it would fall back to a generic implementation like

   for k in d:
  v = d[k]:
 ...

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Stephen J. Turnbull
Greg Ewing writes:

 > Maybe what's wanted is a function analogous to enumerate() for
 > mappings instead of sequences. Picking a semi-arbitrary name
 > for now:
 > 
 > for k, v in tabulate(d):

I thought this already existed in six, though, with a name that is
familiar to Python 2 programmers and not requiring a mental gear
change to recall semantics:

for k,v in six.iteritems(d):

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] subprocess.Popen and win32

2014-04-19 Thread David Aguilar
Hi,

I just joined python-dev because I found the need to add some code to
paper over python3's subprocess API, and I'm wondering whether I'm
missing something.

On python2 and python3, the (only?) way to get utf-8 arguments to
subprocess was to ensure that all unicode strings are encoded into
bytes before subprocess sees them. This has worked for a long time
(currently compatible across python2 and 3).

On python3, this still works for normal platforms, but on windows we
can't pass a list of byte strings. We have to pass a list of unicode
strings.

This means that the application code ends up needing to do this:
https://github.com/git-cola/git-cola/commit/1109aeb4354c49931d9b0435d2b7cfdc2d5d6966

basically,

def start_command(cmd):
if sys.platform == 'win32':
# Python on windows always goes through list2cmdline() internally inside
# of subprocess.py so we must provide unicode strings here otherwise
# Python3 breaks when bytes are provided.
cmd = [decode(c) for c in cmd]
else:
cmd = [encode(c) for c in cmd]
 return subprocess.Popen(cmd)

That seems broken to me, so I wonder if this is a bug in the way
python3 is handling Popen with list-of-bytestring on win32?

I'm not a windows user, but I was able to install python3 under wine
and the same traceback happens without the paper bag fix. This is what
the traceback looks like; it dies in list2cmdline (which I am not
calling directly, Popen does it under the covers):

File "E:\Program Files
(E)\git-cola\share\git-cola\lib\cola\core.py", line 109, in
start_command
universal_newlines=universal_newlines)
File "C:\Python32\lib\subprocess.py", line 744, in __init__
restore_signals, start_new_session)
File "C:\Python32\lib\subprocess.py", line 936, in _execute_child
args = list2cmdline(args)
File "C:\Python32\lib\subprocess.py", line 564, in list2cmdline
needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: Type str doesn't support the buffer API

This is an issue for folks that use python to write cross-platform
code. The unix code paths expect list-of-bytes, but win32 only expects
list-of-unicode, which pushes the burden onto the application
programmer.

It's my opinion that the win32 code path on python3 is the odd man
out. If it allowed list-of-bytes like python2/win32 and python2+3/unix
then this wouldn't be an issue.

Is this an actual problem, or is it something that should be handled
by application-level code as I've done?

Thanks,
-- 
David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Steven D'Aprano
On Sat, Apr 19, 2014 at 10:38:39AM -0400, Nick Coghlan wrote:
> On 19 Apr 2014 00:27, "Steven D'Aprano"  wrote:
> >
> > On Fri, Apr 18, 2014 at 10:31:29PM -0400, Nick Coghlan wrote:
> > > After spending some time talking to the folks at the PyCon Twisted
> > > sprints, they persuaded me that adding back the iterkeys/values/items
> > > methods for mapping objects would be a nice way to eliminate a key
> > > porting hassle for them (and likely others), without significantly
> > > increasing the complexity of Python 3.
> >
> > It would also considerable add to the cruft of Python 3. One motive for
> > going through the pain of Python 2 to 3 migration was to remove cruft.
> > Adding it back in again is not just an aid to porting but actively
> > making Python 3 a worse (well, "less better") experience.
> 
> The cruft is unavoidable in this case. The decision we face is *where the
> cruft lives*, and *how much work is involved* in creating that cruft.

"How much work" is "very little". This problem is not of the same 
magnitude as trying to deal with unicode literals in a polyglot module, 
or the other string/bytes issues. A lot of the time, we don't even care 
whether iterating over dict.foo() gives a list, an iterator or a view. 
For the times we do care, it isn't hard to use a helper.

As for "where the cruft lives", that's the crux of the matter. In my 
opinion, the decision hinges on this question:

   Are iterkeys(), iteritems() and itervalues() the new, preferred 
   APIs for keys(), items() and values(), with the old APIs being 
   kept only for backward compatibility?

If so, then adding them to the language is certainly the right place. 
The old APIs could be deprecated, or even just left with a note in the 
docs that they aren't yet formerly deprecated but will be some day, in 
the meantime the new iterator-based APIs are preferred.

But if the keys(), items() and values() view-based APIs remain the 
preferred API, then I don't believe the backwards-compatibility layer 
belongs in the language. I believe it belongs as an external library, or 
even just a few helpers on an ad hoc basis.

If this is being driven by Twisted, I think that the onus needs to be on 
them to demonstrate how it will help them. Their official plans are to 
support Python 3.3, and they seem to have made a lot of progress towards 
it:

http://twistedmatrix.com/trac/wiki/Plan/Python3

so unless they drop 3.3 and 3.4 (do we want to encourage that?) 
this change won't even help them.


> Status quo: we preserve the "purity" of the Python 3 mapping API, and
> require every developer making the transition from Python 2 to Python 3
> replace every occurrence of these methods with a helper function that is
> not idiomatic code in either language.

That is part of the cost of writing polyglot code. It's messy. But 
that's not new, and it's not unique to 2+3 polyglot code, it happens 
every time a new feature is added or removed in a point release.


> My proposal: we add three trivial helper methods to the Python 3 mapping
> API. For the cost of 3 additional methods that are easily explained in
> terms of combining a builtin with other existing methods, a whole pile of
> work just evaporates for affected segments of the community.

And a whole lot of additional work suddenly appears for *different* 
affected segments of the community, e.g. educators, writers.

This is where the standard objections to any new language feature come 
out. Just because this feature is being added for the benefit of 
transitional 2+3 polyglot code doesn't render these objections 
irrelevant. I trust I don't need to go through the usual list.

By the way, I earlier suggested adding iter* and immediately deprecating 
them, but I don't believe you commented on that. Even if they are never 
removed, the deprecation warning would be a very strong signal that they 
really are only added as an aid to writing 2+3 code, and are not 
intended as long-term language features. Writers can then ignore the 
iter* API, or at least relegate it to an appendix, and educators might 
not be quite able to ignore it but they can at least say "don't use 
them" in good conscience. The dict iteration issue for polygot code now 
becomes easy: just silence the warning, and you're good to go for at 
least three more years, and likely longer.

I think immediate deprecation would be a reasonable compromise position, 
and would like to hear your thoughts on that.


> > So while I'm sympathetic to wanting to ease the 2/3 transition, even at
> > the expense of re-introducing some cruft when unavoidable, I don't think
> > that the difficulty of dealing with dict items|iteritems|viewitems etc.
> > justifies re-adding cruft.
> 
> The cruft will be there regardless, the only question is whether it exists
> in the definition of Python 3 or is distributed across all of the single
> source projects supporting both 2 & 3.

The same reasoning applies to every change between 2 and 3. If 
dict.iter* me

Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Terry Reedy

On 4/19/2014 10:52 AM, Guido van Rossum wrote:

Does everyone involved know that "for x in d.iterkeys()" is equivalent
to "for x in d"


Looking at uses I found by searching code.ohloh.net, the answer is 
either 'No, people sometimes add a redundant .iterkeys()' or 'people are 
writing non-dict mapping classes for which it is not redundant (perhaps 
because their custom class iterates by items rather than keys by 
default)'. I could not tell from the quoted snippet.



and works the same in Python 2 and 3? Similarly,
"list(d)" is a simple, fast way to spell the Python 2 semantics of
"d.keys()"  that works in both versions (but I doubt it is much needed
-- usually the actual code follows up with sorting, so you should use
sorted(d)).

This doesn't solve itervalues() and iteritems() but I expect those are
less common,


ohloh gives about 77,000 python hits for iteritems and 16,000 for 
itervalues. A large fraction of itervalue hits are definitions rather 
than uses, often from a compat.py (is this from six?)


if sys.version_info[0] >= 3:
text_type = str
string_types = str,
iteritems = lambda o: o.items()
itervalues = lambda o: o.values()
izip = zip

else:
text_type = unicode
string_types = basestring,
iteritems = lambda o: o.iteritems()
itervalues = lambda o: o.itervalues()
from itertools import izip

This is three hits for iteritems and three for itervalues and none for 
the unneeded iterkeys. My guess is that there are 5000 itervalue uses 
and 7 iteritem uses.


There are 1,500,000 python hits for 'for', some unknown fraction of 
which are 'for key in somedict' or 'for key in somedict.keys()'.  There 
are 13000 for iterkeys. As noted above, this is *not* inflated by 3 hits 
for each use of compat.py. I think 10% or 15 iterations by key might 
be a reasonable guess.


There are other definition sets that include iterkeys or that define 
functions that wrap all three bound methods for a particular dict.


iterkeys = lambda: d.iterkeys() # py2
iterkeys = lambda: d.keys()  # py3


and "for x, y in d.iteritems(): " is rewritten nicely as

   for x in d:
 y = d[x]
 

If there is a measurable slowdown in the latter I would be totally okay
with some kind of one-element cache for the most recent lookup.


About three weeks ago, Raymond opened http://bugs.python.org/issue21101 
with this claim: "It is reasonably common to make two successive 
dictionary accesses with the same key."  I proposed a specialized 
caching as an alternative to adding new C API functions.


Using the iteritems function, there is one simple, extra function call 
for the entire loop. If the body of the loop takes at lease as long as 
that one call, the extra time is a non-issue if the dict has more than, 
say, 20 items.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

2014-04-19 Thread Ethan Furman

Thank you for taking the time to write this up, Nick.

However, I am -1 on it.  One of the allures of Python 3 is the increase in simplicity and elegance.  Restoring cruft 
does not help with that.  Python 2 idioms that get restored to Python 3 must have real value: unicode literals, 
wire-protocol interpolations -- I don't feel that this comes any where close to meeting that bar.


--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com