Re: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous / Mutable vs Immutable

[Jeff Allen]

The "think of tuples like a struct in C" explanation immediately 
reminded me that ...


On 16/04/2014 21:42, Taavi Burns wrote (in his excellent notes from the 
language summit):

  The demographics have changed. How do
we change the docs and ecosystem to avoid the assumption that Python
programmers already know how to program in C?
Good question. My version was going to be that if you are dealing with 
tuples of mixed data like (name, age, shoesize), inserting something or 
sorting, in the way a list can, would confuse your code. A list, you 
almost always iterate over, to do the same thing with each member, and 
that only works if they are the same type of thing.


Then I realised David Beazley had explained this (but better), starting 
in the Tuples section of his "Python Essential Reference". With 
permission, this could perhaps be adopted wherever it best fits in the 
documentation.


Jeff Allen

On 17/04/2014 20:49, Leandro Pereira de Lima e Silva wrote:


This looks like an issue to be addressed at PEP-8 since it looks like 
a styling issue.


I haven't seen any other recommendations there on how to use a certain 
data structure, though.


Cheers, Leandro

Em 17/04/2014 16:24, "Guido van Rossum" > escreveu:


It's definitely something that should be put in some
documentation, probably at the point when people have learned
enough to be designing their own programs where this issue comes
up -- before they're wizards but well after they have learned the
semantic differences between lists and tuples.


On Thu, Apr 17, 2014 at 11:49 AM, Brett Cannon mailto:bcan...@gmail.com>> wrote:



On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e
Silva mailto:leandro...@cpti.cetuc.puc-rio.br>> wrote:

Hello there!

I've stumbled upon this discussion on python-dev about
what the choice between using a list or a tuple is all
about in 2003:
1.
https://mail.python.org/pipermail/python-dev/2003-March/033962.html
2.
https://mail.python.org/pipermail/python-dev/2003-March/034029.html

There's a vague comment about it on python documentation
but afaik there the discussion hasn't made into any PEPs.
Is there an understanding about it?


Think of tuples like a struct in C, lists like an array.
That's just out of Guido's head so I don't think we have ever
bothered to write it down somewhere as an important
distinction of the initial design that should be emphasized.

___
Python-Dev mailing list
Python-Dev@python.org 
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org




-- 
--Guido van Rossum (python.org/~guido )






___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Language Summit notes

[Kristján Valur Jónsson]

Here, a week later, are some of my thoughts from the summit, for the record:

2.8:
The issue of a hyptothetical 2.8 never fails to entertain.  However, I noticed 
that there seem to be at least two distinct missions of such a thing.

1.   An aid in the conversion from 2.x series to 3.x series.  Enabling a 
bunch of warnings and such by default.  Perhaps allowing 3.x syntax in some 
places without fuss.  The problem with this idea is that it is pointless.  Why 
would anyone want to upgrade from 2.7 to 2.8 if all they get is some new 
warnings for 3.x?  If people are willing to make a version upgrade just to get 
new warnings (i.e. no immediate feature benefit) they might as well go directly 
to 3.x and be done with it.

2.   Feature enhancement to 2.8.  Take a robust and popular version of 
python and add some of the language goodies that have been added to 3.x and 
that don’t have an inherent 3.x aspect.  Yield from.  New exception model.  
Stdlib enhancements such as futures.   The argument goes like this:  We have a 
very popular platform out there with lots of momentum.  People want incremental 
enhancements to it.  Why not give them what they want?  Bread and games and all 
that?  A Rockband cannot stay cooped up in a studio producing experimental 
concept albums all the time.  That is death.  Sometimes it needs to go on tour 
and play old hits for the fans!
3.5 features
When asked what should we aim for in 3.5, there were mostly some very minor 
incremental changes suggested, IIRC.  In my opinion, the reason 3.x has not 
caught on is that there is no real carrot there.  There is no new vision, no 
killer feature.  Nothing that a programmer sees and makes him say “Yeah! I want 
to program my next project using this feature, it will be super!”.
In my opinion we should be thinking more boldly.  Either for 3.x or for a 
version 4.  We should be taking the language to a new level.  Thinking about 
evolving the language.  New paradigms.   Look at what C# is doing, with each 
language revision.  Look at Go.  I’m no CS but here are some ideas on stuff we 
could visit:

1.   Code blocks as a core language construct.  Re-implement context 
managers as block executors.  We shouldn’t let details such as syntax questions 
distract us.   That’s like saying that we can’t  eat spaghetti because our 
Italian is so poor.  Proper code blocks would open up new avenues for 
exploration of expressability and paradigms.

2.   Concurrency primitives built into the language.  Again, see C# with 
its “async” keyword  (a feature we’ve experimented with in stacklesslib, see 
e.g. stacklesslib.async in https://bitbucket.org/stackless-dev/stacklesslib ).  
Look at Go with its channels and more importantly, the select feature.  ( see 
goless, http://goless.readthedocs.org/en/latest/index.html a 2014 sprint 
project).  Don’t get distracted by the GIL.  Concurrency is as much about 
orchestration of operations as it is about parallel execution of code.  Let’s 
learn from the success of stackless, gevent, go, and build on top of it by 
absorbing tried and tested research from more than 30 years of CS.
These are the immediate ideas rolling off the top of my head.  Notice how I 
don’t mention “removing the GIL” here since that is not a “language feature” as 
such, not something inspiring new thinking and invention.  Of course a non-GIL 
implementation is also desirable, even if it would involve completely 
rethinking the C API.  For a version 4 of python.  But I think we thinking 
beyond that, even.

Let’s keep on truckin’!

K


From: Python-Dev [mailto:python-dev-bounces+kristjan=ccpgames@python.org] 
On Behalf Of Guido van Rossum
Sent: 10. apríl 2014 01:08
To: Python-Dev
Subject: [Python-Dev] Language Summit notes

To anyone who took notes at the language summit at PyCon today, even if you 
took them just for yourself, would you mind posting them here? It would be good 
to have some kind of (informal!) as soon as possible, before we collectively 
forget. You won't be held responsible for correctness.
)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 16 Apr 2014 21:03, "R. David Murray"  wrote:
> There is an 'Installing Python on Windows' link further down the google
> results that links to a fairly good page from python-guide.org, whose
> first link lets you download the 2.7.6 msi.  I guess that's the 32
> bit version.  But it then tells you go to python.org to make sure you
> get the latest release, and tells you to click on a link that doesn't
> exist any more (the "windows installer" link).
>
> So, yeah.
>
> Usability.

As part of thrashing out the respective distribution ecosystem roles
of pip and conda (still a work in progress), we're at least converging
on the notion that there are actually now *two* main ways of consuming
Python: as a "software integrator" (the way most of us have
traditionally consumed it, and the way that dominates most project
documentation outside the scientific Python community) and as an "end
user" (the way Linux system administrators have long consumed it, and
the way scientists, financial analysts and folks just learning Python
are likely best off consuming it).

Making these different personas explicit is a process that has barely
begun (this email is mostly based on some conversations I had in
person at PyCon and via email during the sprints), but here's the gist
(based on listing examples):

Software integrators:

* Linux distributions and other operating system vendors
* Sumo redistributions (commercial or otherwise)
* "Python based environments" (PTVS, Enthought Canopy, wakari.io,
Python Anywhere, etc)
* Software-as-a-Service developers
* Device manufacturers
* PC OEMs
* creators of corporate "Standard Operating Environment" definitions
* System integrators (IBM, Boeing et al)
* Application developers (from simple CLI tools to OpenStack)

End users:
* system administrators
* scientists (whether in academia or corporations)
* financial analysts
* engineers
* data miners
* graphic artists
* animation designers
* <<< Folks trying out Python for the first time >>>

For end users, Python is likely consumed as *part of something else*.
For Linux admins, it's a way of scripting the system, for graphic
artists and animators, it's likely embedded as part of a tool like
Blender or Maya, for scientists, financial analysts, engineers and
data miners, it likely makes sense to use it as part of a larger
visualisation environment like IPython Notebook, Python (x, y),
Enthought Canopy, or a hosted system like Wakari or Python Anywhere.

Folks just learning Python are also in the latter list, and we
currently ask them (on the python.org download pages) to jump straight
into the "software integrator" role to get their environment set up,
rather than setting out to impress them by recommending one of the
pre-integrated sumo distributions that offers quick and easy access to
powerful visualisation and data analysis tools. While providing access
to the CPython default interactive prompt in the browser is cool, it's
less impressive as the default experience we provide after someone has
downloaded and installed the interpreter. Instead, we likely want to
*really* impress them by making it easy for them to explore the full
power of things like IPython Notebooks. My own current preferred
approach for that is the fully open source "Anaconda" distribution
from Continuum Analytics, specifically because it *is* fully open
source, and you can "pip install conda" to bootstrap their package
manager in other contexts. And, importantly, because conda
environments can *manage the Python runtime itself*, it is able to
work more consistently across different platforms than the upstream
tools by reliably isolating itself from the system Python on POSIX
platforms. It should even by possible to get conda to manage alternate
Python implementations like Stackless, PyPy, Jython, IronPython, etc
(although I don't believe anyone is using it that way as yet).

However, conda *isn't* optimised for the software integrator use case
- the additional work it does to improve cross platform consistency
actually *gets in the way*, when you're trying to integrate Python
closely with a larger system (like a Linux distribution), and "a large
set of automatically provided libraries" isn't actually a feature in
that context. While conda does offer some options (like miniconda) to
make that kind of integration task easier, I strongly believe that
realm is better handled by consuming CPython and the standard library,
along with pip and related tools, directly, and accepting the
additional platform specific details that come along with that.

Making this "software integrator" and "end user" split deliberately
and consciously, and pushing the former toward consuming things in a
more DIY fashion, and the latter towards a pre-assembled sumo
distribution should help greatly in allowing us to better balance the
starkly different needs of the two groups, and provide an extremely
high quality "out of the box" experience for new users, while still
allowing software integrators to

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2014-04-11 - 2014-04-18)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open4579 (+12)
  closed 28493 (+88)
  total  33072 (+100)

Open issues with patches: 2118 


Issues opened (68)
==

#17160: test_urllib2net fails
http://bugs.python.org/issue17160  reopened by ddve...@ucar.edu

#17861: put opcode information in one place
http://bugs.python.org/issue17861  reopened by twouters

#21201: Uninformative error message in multiprocessing.Manager()
http://bugs.python.org/issue21201  opened by wojtekwalczak

#21202: Naming a file` io.py` causes cryptic error message
http://bugs.python.org/issue21202  opened by madison.may

#21204: multiprocessing example does not work on Windows
http://bugs.python.org/issue21204  opened by jmaki

#21205: Unable to make decorated generator object to inherit generator
http://bugs.python.org/issue21205  opened by SzieberthAdam

#21207: urandom persistent fd - not re-openned after fd close
http://bugs.python.org/issue21207  opened by kwirk

#21211: pkgutil.find_loader() raises ImportError instead of returning 
http://bugs.python.org/issue21211  opened by eric.snow

#21213: Memory bomb by incorrect custom serializer to json.dumps
http://bugs.python.org/issue21213  opened by saaj

#21216: getaddrinfo is wrongly considered thread safe on linux
http://bugs.python.org/issue21216  opened by Julien.Palard

#21217: inspect.getsourcelines finds wrong lines when lambda used argu
http://bugs.python.org/issue21217  opened by ballingt

#21220: Enhance obmalloc allocation strategy
http://bugs.python.org/issue21220  opened by kristjan.jonsson

#21221: Minor struct_time documentation bug
http://bugs.python.org/issue21221  opened by lurchman

#21224: BaseHTTPRequestHandler, update the protocol version to http 1.
http://bugs.python.org/issue21224  opened by matrixise

#21225: io.py: Improve docstrings for classes
http://bugs.python.org/issue21225  opened by akuchling

#21226: PyImport_ExecCodeModuleObject not setting module attributes
http://bugs.python.org/issue21226  opened by trevor3

#21227: Decimal class error messages for integer division aren't good
http://bugs.python.org/issue21227  opened by leewz

#21228: Missing enumeration of HTTPResponse Objects methods of urllib.
http://bugs.python.org/issue21228  opened by EvensF

#21230: imghdr does not accept adobe photoshop mime type
http://bugs.python.org/issue21230  opened by faiz

#21231: Issue a python 3 warning when old style classes are defined.
http://bugs.python.org/issue21231  opened by alex

#21232: Use of '1' instead of 'True' as 'splitlines' argument in diffl
http://bugs.python.org/issue21232  opened by jayanthkoushik

#21233: Add *Calloc functions to CPython memory allocation API
http://bugs.python.org/issue21233  opened by njs

#21235: importlib's spec module create algorithm is not exposed
http://bugs.python.org/issue21235  opened by aronacher

#21236: patch to use cabinet.lib instead of fci.lib (fixes build with 
http://bugs.python.org/issue21236  opened by mst

#21237: Update Python 2/3 porting HOWTO's suggestion for dealing with 
http://bugs.python.org/issue21237  opened by brett.cannon

#21240: Add an abstactmethod directive to the Python ReST domain
http://bugs.python.org/issue21240  opened by eric.snow

#21242: Generalize configure check for working Python executable
http://bugs.python.org/issue21242  opened by brett.cannon

#21243: Auto-generate exceptions.c from a Python file
http://bugs.python.org/issue21243  opened by brett.cannon

#21247: test_asyncio: test_subprocess_send_signal hangs on Fedora buil
http://bugs.python.org/issue21247  opened by opoplawski

#21249: removing pythonXY.zip from sys.path results in additional test
http://bugs.python.org/issue21249  opened by doko

#21250: sqlite3 doesn't have unit tests for 'insert or [algorithm]' fu
http://bugs.python.org/issue21250  opened by Alex.Lord

#21251: Standard library trace module crashes with exception
http://bugs.python.org/issue21251  opened by mkolman

#21252: Lib/asyncio/events.py has tons of docstrings which are just "X
http://bugs.python.org/issue21252  opened by alex

#21253: Difflib.compare() crashes when sequences contain little or no 
http://bugs.python.org/issue21253  opened by nnja

#21254: PropertyMock refuses to raise AttributeErrror as a side effect
http://bugs.python.org/issue21254  opened by michael.foord

#21255: Attaching a PropertyMock records calls
http://bugs.python.org/issue21255  opened by michael.foord

#21256: Sort keyword arguments in mock _format_call_signature
http://bugs.python.org/issue21256  opened by michael.foord

#21257: Document parse_headers function of http.client
http://bugs.python.org/issue21257  opened by orsenthil

#21258: Add __iter__ support for mock_open
http://bugs.python.org/issue21258  opened by michael.foord

#21259: replace "except: pass" by "except

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Guido van Rossum]

Could I summarize that as "software integrators build from source, while
end users use an installer"? And the rest of the discussion is about which
installer (in the widest sense of the word) to recommend, where the choices
include Linux vendor distros, sumo Python distros, Mac/Win installers, as
well as fine-grained tools like apt-get (Linux vendor specific) and pip and
conda (Python specific)? And perhaps conda is pushing the boundaries
because it also tries to manage things that aren't strictly Python, but
useful for scientists?


On Fri, Apr 18, 2014 at 8:58 AM, Nick Coghlan  wrote:

> On 16 Apr 2014 21:03, "R. David Murray"  wrote:
> > There is an 'Installing Python on Windows' link further down the google
> > results that links to a fairly good page from python-guide.org, whose
> > first link lets you download the 2.7.6 msi.  I guess that's the 32
> > bit version.  But it then tells you go to python.org to make sure you
> > get the latest release, and tells you to click on a link that doesn't
> > exist any more (the "windows installer" link).
> >
> > So, yeah.
> >
> > Usability.
>
> As part of thrashing out the respective distribution ecosystem roles
> of pip and conda (still a work in progress), we're at least converging
> on the notion that there are actually now *two* main ways of consuming
> Python: as a "software integrator" (the way most of us have
> traditionally consumed it, and the way that dominates most project
> documentation outside the scientific Python community) and as an "end
> user" (the way Linux system administrators have long consumed it, and
> the way scientists, financial analysts and folks just learning Python
> are likely best off consuming it).
>
> Making these different personas explicit is a process that has barely
> begun (this email is mostly based on some conversations I had in
> person at PyCon and via email during the sprints), but here's the gist
> (based on listing examples):
>
> Software integrators:
>
> * Linux distributions and other operating system vendors
> * Sumo redistributions (commercial or otherwise)
> * "Python based environments" (PTVS, Enthought Canopy, wakari.io,
> Python Anywhere, etc)
> * Software-as-a-Service developers
> * Device manufacturers
> * PC OEMs
> * creators of corporate "Standard Operating Environment" definitions
> * System integrators (IBM, Boeing et al)
> * Application developers (from simple CLI tools to OpenStack)
>
> End users:
> * system administrators
> * scientists (whether in academia or corporations)
> * financial analysts
> * engineers
> * data miners
> * graphic artists
> * animation designers
> * <<< Folks trying out Python for the first time >>>
>
> For end users, Python is likely consumed as *part of something else*.
> For Linux admins, it's a way of scripting the system, for graphic
> artists and animators, it's likely embedded as part of a tool like
> Blender or Maya, for scientists, financial analysts, engineers and
> data miners, it likely makes sense to use it as part of a larger
> visualisation environment like IPython Notebook, Python (x, y),
> Enthought Canopy, or a hosted system like Wakari or Python Anywhere.
>
> Folks just learning Python are also in the latter list, and we
> currently ask them (on the python.org download pages) to jump straight
> into the "software integrator" role to get their environment set up,
> rather than setting out to impress them by recommending one of the
> pre-integrated sumo distributions that offers quick and easy access to
> powerful visualisation and data analysis tools. While providing access
> to the CPython default interactive prompt in the browser is cool, it's
> less impressive as the default experience we provide after someone has
> downloaded and installed the interpreter. Instead, we likely want to
> *really* impress them by making it easy for them to explore the full
> power of things like IPython Notebooks. My own current preferred
> approach for that is the fully open source "Anaconda" distribution
> from Continuum Analytics, specifically because it *is* fully open
> source, and you can "pip install conda" to bootstrap their package
> manager in other contexts. And, importantly, because conda
> environments can *manage the Python runtime itself*, it is able to
> work more consistently across different platforms than the upstream
> tools by reliably isolating itself from the system Python on POSIX
> platforms. It should even by possible to get conda to manage alternate
> Python implementations like Stackless, PyPy, Jython, IronPython, etc
> (although I don't believe anyone is using it that way as yet).
>
> However, conda *isn't* optimised for the software integrator use case
> - the additional work it does to improve cross platform consistency
> actually *gets in the way*, when you're trying to integrate Python
> closely with a larger system (like a Linux distribution), and "a large
> set of automatically provided libraries" isn't actually a feature in
> that context.

Re: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous / Mutable vs Immutable

[Nick Coghlan]

On 18 April 2014 04:21, Jeff Allen  wrote:
>
> The "think of tuples like a struct in C" explanation immediately reminded me
> that ...
>
> On 16/04/2014 21:42, Taavi Burns wrote (in his excellent notes from the
> language summit):
>
>  The demographics have changed. How do
> we change the docs and ecosystem to avoid the assumption that Python
> programmers already know how to program in C?
>
> Good question. My version was going to be that if you are dealing with
> tuples of mixed data like (name, age, shoesize), inserting something or
> sorting, in the way a list can, would confuse your code. A list, you almost
> always iterate over, to do the same thing with each member, and that only
> works if they are the same type of thing.
>
> Then I realised David Beazley had explained this (but better), starting in
> the Tuples section of his "Python Essential Reference". With permission,
> this could perhaps be adopted wherever it best fits in the documentation.

This is the kind of "competence to mastery" transition my "Python
User's Reference" was designed to cover, but the book contract fell
through, and I was never able to interest anyone in rescuing the
material from the ODF manuscript and migrating it to Sphinx after I
donated it to the PSF:
http://svn.python.org/view/sandbox/trunk/userref/

(It's also all 2.5 era docs, so would need a fair bit of work to bring
it up to date for 2.7 and 3.4)

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Paul Moore]

On 18 April 2014 16:58, Nick Coghlan  wrote:
> As part of thrashing out the respective distribution ecosystem roles
> of pip and conda (still a work in progress), we're at least converging
> on the notion that there are actually now *two* main ways of consuming
> Python: as a "software integrator" (the way most of us have
> traditionally consumed it, and the way that dominates most project
> documentation outside the scientific Python community) and as an "end
> user" (the way Linux system administrators have long consumed it, and
> the way scientists, financial analysts and folks just learning Python
> are likely best off consuming it).
>
> Making these different personas explicit is a process that has barely
> begun (this email is mostly based on some conversations I had in
> person at PyCon and via email during the sprints), but here's the gist
> (based on listing examples):

Interesting perspective. However, I'm not convinced it's complete.
Specifically, there's one group of people who I encounter relatively
often, who don't seem to me to fit well into either category you're
proposing. That is, (Windows in my experience, but maybe Linux as
well) users who want to write "simple scripts" and for whom batch
files or similar are not sufficient. Such people typically don't have
the sort of "single application area" focus that your "end user"
category seems to imply, but on the other hand they don't really fit
the "software integrators" role in the sense of necessarily being
comfortable setting up their own development environment.

I worry that your classification risks ignoring that group (maybe
because Unix users are well served with other alternatives than Python
for this type of task, or because on Unix "use the system Python" is
the right answer).

Your list of "end user" targeted distributions seem to be limited to:

  - Linux distribution vendors
  - Vendors focused on the essentially scientific community (in the
broadest sense)
  - Embedded Python

That's very far from being complete coverage of all the people *I'd*
like to be able to recommend Python to. Specifically, unless we're not
interested in "generic" Windows users, I think we need to offer *some*
form of equivalent of the OS-packaged Python on Linux for Windows
users. That's what the python.org builds, plus pip, wheels and PyPI,
give for Windows users now. Hmm, if we assume that supporting that
remains a priority, is what you're really saying that we *don't* try
to extend that to work for Linux/OSX, as doing so competes with the OS
vendors - but rather we see python.org binaries and binary
infrastructure like wheels as being focused on the Windows user
experience?

(I wish I'd been at PyCon, this would have been a very interesting
discussion to have face to face. Email isn't ideal for this...)

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Daniel Holth]

On Fri, Apr 18, 2014 at 12:55 PM, Paul Moore  wrote:
> On 18 April 2014 16:58, Nick Coghlan  wrote:
>> As part of thrashing out the respective distribution ecosystem roles
>> of pip and conda (still a work in progress), we're at least converging
>> on the notion that there are actually now *two* main ways of consuming
>> Python: as a "software integrator" (the way most of us have
>> traditionally consumed it, and the way that dominates most project
>> documentation outside the scientific Python community) and as an "end
>> user" (the way Linux system administrators have long consumed it, and
>> the way scientists, financial analysts and folks just learning Python
>> are likely best off consuming it).
>>
>> Making these different personas explicit is a process that has barely
>> begun (this email is mostly based on some conversations I had in
>> person at PyCon and via email during the sprints), but here's the gist
>> (based on listing examples):
>
> Interesting perspective. However, I'm not convinced it's complete.
> Specifically, there's one group of people who I encounter relatively
> often, who don't seem to me to fit well into either category you're
> proposing. That is, (Windows in my experience, but maybe Linux as
> well) users who want to write "simple scripts" and for whom batch
> files or similar are not sufficient. Such people typically don't have
> the sort of "single application area" focus that your "end user"
> category seems to imply, but on the other hand they don't really fit
> the "software integrators" role in the sense of necessarily being
> comfortable setting up their own development environment.
>
> I worry that your classification risks ignoring that group (maybe
> because Unix users are well served with other alternatives than Python
> for this type of task, or because on Unix "use the system Python" is
> the right answer).
>
> Your list of "end user" targeted distributions seem to be limited to:
>
>   - Linux distribution vendors
>   - Vendors focused on the essentially scientific community (in the
> broadest sense)
>   - Embedded Python
>
> That's very far from being complete coverage of all the people *I'd*
> like to be able to recommend Python to. Specifically, unless we're not
> interested in "generic" Windows users, I think we need to offer *some*
> form of equivalent of the OS-packaged Python on Linux for Windows
> users. That's what the python.org builds, plus pip, wheels and PyPI,
> give for Windows users now. Hmm, if we assume that supporting that
> remains a priority, is what you're really saying that we *don't* try
> to extend that to work for Linux/OSX, as doing so competes with the OS
> vendors - but rather we see python.org binaries and binary
> infrastructure like wheels as being focused on the Windows user
> experience?
>
> (I wish I'd been at PyCon, this would have been a very interesting
> discussion to have face to face. Email isn't ideal for this...)

One more group that I find interesting is application users. These
people should not need to notice that Python is in use at all, in
contrast to the "build a virtualenv / install / pass through fire and
death / use" workflow that is sometimes promoted. They are well served
by good tools that make single-file zip distributions or py2exe etc.
from a collection of wheels or sdists. Application users are using
Python because a best-in-class program is written in Python and not
because a Python program integrates better with other Python
libraries.


Conda is interesting because it is a system package manager, except it
installs everything into what they sometimes describe as "C-level
virtualenvs" rather than /. They've come up with a design that works
well with Python programs but isn't particularly Python-specific at
all.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Language Summit notes

[Nick Coghlan]

On 18 April 2014 08:04, Kristján Valur Jónsson  wrote:
> Here, a week later, are some of my thoughts from the summit, for the record:
> 1.   An aid in the conversion from 2.x series to 3.x series.  Enabling a
> bunch of warnings and such by default.  Perhaps allowing 3.x syntax in some
> places without fuss.  The problem with this idea is that it is pointless.
> Why would anyone want to upgrade from 2.7 to 2.8 if all they get is some new
> warnings for 3.x?  If people are willing to make a version upgrade just to
> get new warnings (i.e. no immediate feature benefit) they might as well go
> directly to 3.x and be done with it.

Right, this is much better handled through tools like linters, import
hooks, custom interpreters, additional support modules, etc. I think
http://python-future.org/ represents the current state of the art, but
there are still a lot of opportunities for improvements here.

> 2.   Feature enhancement to 2.8.  Take a robust and popular version of
> python and add some of the language goodies that have been added to 3.x and
> that don’t have an inherent 3.x aspect.  Yield from.  New exception model.
> Stdlib enhancements such as futures.   The argument goes like this:  We have
> a very popular platform out there with lots of momentum.  People want
> incremental enhancements to it.  Why not give them what they want?  Bread
> and games and all that?  A Rockband cannot stay cooped up in a studio
> producing experimental concept albums all the time.  That is death.
> Sometimes it needs to go on tour and play old hits for the fans!

Do you know how much work a new Python 2.x release creates for people?
All the redistributors have to update, books get outdated, a new
wrinkle gets added to the compatibility matrix for everyone. A new
Python 2.x release is simple untenable at this point in the transition
- it's a *massively* expensive way to achieve things that can be
achieved more cheaply in other ways.

Take yield from, for example. Hy is able to compile *LISP* syntax to
Python AST structures. PEP 380 includes a semantic expansion of yield
from in terms of yield. Is it *really* impossible to get "yield from"
based code running in Python 2.6? Or have people just assumed it's not
possible and never even tried, because the idea of using import hooks
to backport syntax to earlier feature releases is too novel?

(importlib includes tools to make this kind of thing relatively
straightforward, and there's nothing inherently impossible about using
importlib to write import hooks in Python 2 - it's just a matter of
having to do the backport first, and Eric Snow is now seriously
considering doing exactly that. Hit us up on import-sig if you think
that might be interesting to you)

> 3.5 features
>
> When asked what should we aim for in 3.5, there were mostly some very minor
> incremental changes suggested, IIRC.  In my opinion, the reason 3.x has not
> caught on is that there is no real carrot there.  There is no new vision, no
> killer feature.  Nothing that a programmer sees and makes him say “Yeah! I
> want to program my next project using this feature, it will be super!”.

I *really* wish folks from North America, Europe and other regions
where 8-bit encodings can handle their native language and where
Anglicisation of terms to fit them into the ASCII identifier
restriction poses no barrier to communication would stop trotting out
this "no killer feature in Python 3" canard. While it is *possible* to
write internationalised and localised applications in it, Python 2's
Unicode support is so broken that some people can't even run the
interpreter from their home directory because it can't cope with their
username. Python 3 makes Unicode support in the interpreter pervasive
(just as it is in the JVM and CLR), and we've spent the last 5+ years
dealing with the latent Unicode handling defects this uncovered in the
standard library, fixing mistakes we made in the initial transition,
and figuring out how to get the new system to play nice with the POSIX
"text is just bytes in a particular encoding" model, as well as
figuring out what we really lost in the transition when it comes to
binary data manipulation.

If anyone is *ever* tempted to utter the words "Python 3 has no killer
feature" without immediately following it up with the "for me"
qualifier, please go read this post about the creation of a Portuguese
version of Stack Overflow:
http://blog.stackoverflow.com/2014/02/cant-we-all-be-reasonable-and-speak-english/

Late last year, Alex Gaynor made the insightful observation that we
write software either for the users we have or for the users we want.
Python 3 is the software we're writing for the users we want. Python
2.7 long term maintenance, PyPy, standard library backports,
transition libraries and tools, packaging ecosystem improvements,
Python 3 features designed primarily to easy migration from Python 2,
PyPI modules that run on both Python 2 & 3 - that's the software we
and many other p

Re: [Python-Dev] devguide: Add note about Kushal Das' privs

[Antoine Pitrou]

On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
brett.cannon  wrote:
> http://hg.python.org/devguide/rev/c14c8a195fec
> changeset:   686:c14c8a195fec
> user:Brett Cannon 
> date:Mon Apr 14 17:18:37 2014 -0400
> summary:
>   Add note about Kushal Das' privs

I have no objection to Kushal getting commit rights, but is there a
public record of the discussion leading to this decision? I can't find
anything on either python-dev or python-committers (perhaps my search
skills are failing me).

Also, welcome on board, Kushal!

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Language Summit notes

[Antoine Pitrou]

On Fri, 18 Apr 2014 12:04:10 +
Kristján Valur Jónsson  wrote:
> 
> 2.   Feature enhancement to 2.8.  Take a robust and popular version of 
> python and add some of the language goodies that have been added to 3.x and 
> that don’t have an inherent 3.x aspect.  Yield from.  New exception model.  
> Stdlib enhancements such as futures.   The argument goes like this:  We have 
> a very popular platform out there with lots of momentum.  People want 
> incremental enhancements to it.  Why not give them what they want?  Bread and 
> games and all that?  A Rockband cannot stay cooped up in a studio producing 
> experimental concept albums all the time.  That is death.  Sometimes it needs 
> to go on tour and play old hits for the fans!

I don't think we have recent download numbers since the Website
overhaul (do we?), but Python 3 isn't an "experimental concept
language" anymore (it hasn't been since 3.3 or 3.2, I'd say).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] devguide: Add note about Kushal Das' privs

[Benjamin Peterson]

On Fri, Apr 18, 2014, at 11:29, Antoine Pitrou wrote:
> On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
> brett.cannon  wrote:
> > http://hg.python.org/devguide/rev/c14c8a195fec
> > changeset:   686:c14c8a195fec
> > user:Brett Cannon 
> > date:Mon Apr 14 17:18:37 2014 -0400
> > summary:
> >   Add note about Kushal Das' privs
> 
> I have no objection to Kushal getting commit rights, but is there a
> public record of the discussion leading to this decision? I can't find
> anything on either python-dev or python-committers (perhaps my search
> skills are failing me).

Probably in person conversion at PyCon sprints.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] devguide: Add note about Kushal Das' privs

[Brett Cannon]

On Friday, April 18, 2014 2:35:32 PM, Benjamin Peterson 
wrote:
>
> On Fri, Apr 18, 2014, at 11:29, Antoine Pitrou wrote:
> > On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
> > brett.cannon  wrote:
> > > http://hg.python.org/devguide/rev/c14c8a195fec
> > > changeset:   686:c14c8a195fec
> > > user:Brett Cannon 
> > > date:Mon Apr 14 17:18:37 2014 -0400
> > > summary:
> > >   Add note about Kushal Das' privs
> >
> > I have no objection to Kushal getting commit rights, but is there a
> > public record of the discussion leading to this decision? I can't find
> > anything on either python-dev or python-committers (perhaps my search
> > skills are failing me).
>
> Probably in person conversion at PyCon sprints

What Benjamin said; discussions at PyCon amongst several of us.

-Brett

>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/brett%40python.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] devguide: Add note about Kushal Das' privs

[Zachary Ware]

On Fri, Apr 18, 2014 at 1:54 PM, Brett Cannon  wrote:
> On Friday, April 18, 2014 2:35:32 PM, Benjamin Peterson
>  wrote:
>> On Fri, Apr 18, 2014, at 11:29, Antoine Pitrou wrote:
>> > On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
>> > brett.cannon  wrote:
>> > > http://hg.python.org/devguide/rev/c14c8a195fec
>> > > changeset:   686:c14c8a195fec
>> > > user:Brett Cannon 
>> > > date:Mon Apr 14 17:18:37 2014 -0400
>> > > summary:
>> > >   Add note about Kushal Das' privs
>> >
>> > I have no objection to Kushal getting commit rights, but is there a
>> > public record of the discussion leading to this decision? I can't find
>> > anything on either python-dev or python-committers (perhaps my search
>> > skills are failing me).
>>
>> Probably in person conversion at PyCon sprints
>
> What Benjamin said; discussions at PyCon amongst several of us.

Looking at that page, it doesn't look like I was ever added.  Brett,
was it you that enabled me?

Also, welcome Kushal!

-- 
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Language Summit notes

[Donald Stufft]

On Apr 18, 2014, at 2:31 PM, Antoine Pitrou  wrote:

> On Fri, 18 Apr 2014 12:04:10 +
> Kristján Valur Jónsson  wrote:
>> 
>> 2.   Feature enhancement to 2.8.  Take a robust and popular version of 
>> python and add some of the language goodies that have been added to 3.x and 
>> that don’t have an inherent 3.x aspect.  Yield from.  New exception model.  
>> Stdlib enhancements such as futures.   The argument goes like this:  We have 
>> a very popular platform out there with lots of momentum.  People want 
>> incremental enhancements to it.  Why not give them what they want?  Bread 
>> and games and all that?  A Rockband cannot stay cooped up in a studio 
>> producing experimental concept albums all the time.  That is death.  
>> Sometimes it needs to go on tour and play old hits for the fans!
> 
> I don't think we have recent download numbers since the Website
> overhaul (do we?), but Python 3 isn't an "experimental concept
> language" anymore (it hasn't been since 3.3 or 3.2, I'd say).
> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

Fastly logs are stored in Dreamhost so we could make those numbers if they 
aren’t already available.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 12:55, Paul Moore  wrote:
> On 18 April 2014 16:58, Nick Coghlan  wrote:
>> As part of thrashing out the respective distribution ecosystem roles
>> of pip and conda (still a work in progress), we're at least converging
>> on the notion that there are actually now *two* main ways of consuming
>> Python: as a "software integrator" (the way most of us have
>> traditionally consumed it, and the way that dominates most project
>> documentation outside the scientific Python community) and as an "end
>> user" (the way Linux system administrators have long consumed it, and
>> the way scientists, financial analysts and folks just learning Python
>> are likely best off consuming it).
>>
>> Making these different personas explicit is a process that has barely
>> begun (this email is mostly based on some conversations I had in
>> person at PyCon and via email during the sprints), but here's the gist
>> (based on listing examples):
>
> Interesting perspective. However, I'm not convinced it's complete.

Personas are never complete - however, they represent sufficiently
useful archetypes that help guide design decisions. Where alarm bells
need to start ringing is when two personas are far enough apart in
interests and skill levels that it is highly unlikely that one tool
will be able to satisfy both groups of users.

I think that's the situation we find ourselves in with respect to
folks that are already professional programmers and folks that just
need to do some programming as part of another activity.

> Specifically, there's one group of people who I encounter relatively
> often, who don't seem to me to fit well into either category you're
> proposing. That is, (Windows in my experience, but maybe Linux as
> well) users who want to write "simple scripts" and for whom batch
> files or similar are not sufficient. Such people typically don't have
> the sort of "single application area" focus that your "end user"
> category seems to imply, but on the other hand they don't really fit
> the "software integrators" role in the sense of necessarily being
> comfortable setting up their own development environment.

The "single application area" in that case is "Windows scripting", and
it *would* make sense to have a sumo distribution that provided
PyWin32 and other components. That distribution may include other
things as well, though.

> I worry that your classification risks ignoring that group (maybe
> because Unix users are well served with other alternatives than Python
> for this type of task, or because on Unix "use the system Python" is
> the right answer).

I think the conda "end user" approach handles that case pretty well
(while it certainly started with the scientific Python stack, conda
certainly isn't limited to that). So the entry can be "system
administrators" in general, rather than being specific to Linux.

The "lack of integration" issues on Linux and Mac OS X show up because
there *is* a system Python for conda to be separate from. We don't
have that problem on Windows (although integrating with the Python
launcher may be an issue).

> Your list of "end user" targeted distributions seem to be limited to:
>
>   - Linux distribution vendors

As noted above, this category is broader than just Linux - it really
covers any system scripting context.

>   - Vendors focused on the essentially scientific community (in the
> broadest sense)

I'd say "analytical", rather than specifically "scientific". From a
user experience perspective though, the important point is that many
of the folks being targeted *aren't* programmers in the traditional
sense - they just need to do some programming as part of their actual
job. It turns out that's also a useful usability proxy for people just
starting out with programming :)

>   - Embedded Python
>
> That's very far from being complete coverage of all the people *I'd*
> like to be able to recommend Python to. Specifically, unless we're not
> interested in "generic" Windows users,

I think conda covers that "generic Windows user" case as well - that's
why I emphasised the fully open source nature of it. We've also been
having some interesting conversations with Travis about potentially
improving the level of collaboration between the PyPA and conda devs.

> I think we need to offer *some*
> form of equivalent of the OS-packaged Python on Linux for Windows
> users. That's what the python.org builds, plus pip, wheels and PyPI,
> give for Windows users now. Hmm, if we assume that supporting that
> remains a priority, is what you're really saying that we *don't* try
> to extend that to work for Linux/OSX, as doing so competes with the OS
> vendors - but rather we see python.org binaries and binary
> infrastructure like wheels as being focused on the Windows user
> experience?

I'd still like to improve the wheel story for Linux (there are various
benefits of that which accrue to system integrators as well), but I
think the key benefit of promoting both pip *and* con

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Donald Stufft]

On Apr 18, 2014, at 3:18 PM, Nick Coghlan  wrote:

> At this point, however, I'm mainly looking for consensus that there
> *are* two different problems to be solved here, and solving them both
> well in a single tool is likely to be nigh on impossible. (I'm aware
> we're really on the wrong list for that discussion, but I also think
> there's value in getting some broader python-dev awareness of this
> particular topic)

I’m not sure about this? I mean yes those are two different areas, but I’m
not sure about the split between Conda and pip here. As far as I can tell
Conda is useful in the same way apt-get or yum is useful, you get a non
Python specific set of packages (because sometimes things aren’t pure
python) and you also remove a little bit of thinking about versions (although
honestly I think it’s possible to remove most of that for consumers of
packages).

To be quite frank, a lot of the benefit of Conda outside of the “I need 
something
that isn’t strictly Python) is in the fact they can bootstrap compiled packages
whereas pip/wheel/PyPI combination we need to convince authors to upload
their own binary packages (or at least develop something to make it easier
like a build farm).

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] List vs Tuple / Homogeneous vs Heterogeneous / Mutable vs Immutable

[Ezio Melotti]

Hi,

On Thu, Apr 17, 2014 at 10:23 PM, Guido van Rossum  wrote:
> It's definitely something that should be put in some documentation,

see http://bugs.python.org/issue14840 and
https://docs.python.org/3/tutorial/datastructures.html#tuples-and-sequences
:
"""
Though tuples may seem similar to lists, they are often used in
different situations and for different purposes. Tuples are immutable,
and usually contain an heterogeneous sequence of elements that are
accessed via unpacking (see later in this section) or indexing (or
even by attribute in the case of namedtuples). Lists are mutable, and
their elements are usually homogeneous and are accessed by iterating
over the list.
"""

Best Regards,
Ezio Melotti

> probably
> at the point when people have learned enough to be designing their own
> programs where this issue comes up -- before they're wizards but well after
> they have learned the semantic differences between lists and tuples.
>
>
> On Thu, Apr 17, 2014 at 11:49 AM, Brett Cannon  wrote:
>>
>>
>>
>> On Thu Apr 17 2014 at 2:43:35 PM, Leandro Pereira de Lima e Silva
>>  wrote:
>>>
>>> Hello there!
>>>
>>> I've stumbled upon this discussion on python-dev about what the choice
>>> between using a list or a tuple is all about in 2003:
>>> 1. https://mail.python.org/pipermail/python-dev/2003-March/033962.html
>>> 2. https://mail.python.org/pipermail/python-dev/2003-March/034029.html
>>>
>>> There's a vague comment about it on python documentation but afaik there
>>> the discussion hasn't made into any PEPs. Is there an understanding about
>>> it?
>>
>>
>> Think of tuples like a struct in C, lists like an array. That's just out
>> of Guido's head so I don't think we have ever bothered to write it down
>> somewhere as an important distinction of the initial design that should be
>> emphasized.
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ezio.melotti%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 15:39, Donald Stufft  wrote:
>
> On Apr 18, 2014, at 3:18 PM, Nick Coghlan  wrote:
>
>> At this point, however, I'm mainly looking for consensus that there
>> *are* two different problems to be solved here, and solving them both
>> well in a single tool is likely to be nigh on impossible. (I'm aware
>> we're really on the wrong list for that discussion, but I also think
>> there's value in getting some broader python-dev awareness of this
>> particular topic)
>
> I’m not sure about this? I mean yes those are two different areas, but I’m
> not sure about the split between Conda and pip here. As far as I can tell
> Conda is useful in the same way apt-get or yum is useful, you get a non
> Python specific set of packages (because sometimes things aren’t pure
> python) and you also remove a little bit of thinking about versions (although
> honestly I think it’s possible to remove most of that for consumers of
> packages).

You also get the ability to use that same system to update *Python
itself*, regardless of platform. Being notified of and consuming
CPython updates on Windows, as well as consuming alternate versions on
Linux distros, is currently not a well solved problem - with conda,
it's no more complex than updating a PyPI package.

That's one of the most attractive aspects for me - making Python 3.4,
pypy, etc as easy to update with consistent cross platform
instructions as Python packages are.

> To be quite frank, a lot of the benefit of Conda outside of the “I need 
> something
> that isn’t strictly Python) is in the fact they can bootstrap compiled 
> packages
> whereas pip/wheel/PyPI combination we need to convince authors to upload
> their own binary packages (or at least develop something to make it easier
> like a build farm).

Yep, that's a large part of why I think "divide and conquer" is the
way to go here. While it isn't completely accurate (as most SaaS
developers don't want to build C extensions from source) I think
Guido's "build from source" vs "install a pre-built binary"
distinction is still a reasonably good way to characterise it. For a
distro like Fedora (or, even more so, RHEL), we're not going to trust
a binary created by someone else if we can possibly avoid it, so
upstream binaries aren't useful to us, but pip's ability to abstract
away the vagaries of the upcoming metadata 2.0 migration is incredibly
helpful. The other distros are in the same situation (we'll always
feed source tarballs into our own build systems), and ditto for the
conda folks. We need that for all sorts of reasons that potential new
Python users don't care about, but continuing to meet our
requirements, along with the free-for-all that is PyPI makes handling
the binary distribution problem *much* harder for pip.

By contrast, like any other distro, conda doesn't need to boil the
ocean - it just needs to provide a filtered, up to date set of core
packages that work well together. The advantage it has over other
distros is that it is *cross-platform* - it works essentially the same
way on Windows, Mac OS X. Most other package management systems are
either platform specific or can't handle arbitrary binary
dependencies. By being Python-centric (even if not Python specific),
there's also a strong focus on updating the core packages more often
than the Linux distros do.

There's no "always use conda instead of pip" competition here - we
need pip, we need wheels. But I see those as software integrator
focused tools that would need a lot of additional work to provide a
truly spectacularly compelling out of the box experience for new
users. I don't think that's a useful way to expend effort - I think it
makes more sense to work on a separate "here's the fully assembled
environment for using Python as a tool to develop ideas" introduction,
while pushing the "here's how to build your own custom environment
from upstream parts" as a later step in a new user's journey towards
full Python mastery (if they're interested in that path). Making sure
that "pip install foo" does the right thing inside conda environments
(if it doesn't already) should be all that is needed to ensure that
random installation instructions on the internet still work.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Paul Moore]

On 18 April 2014 20:18, Nick Coghlan  wrote:
> At this point, however, I'm mainly looking for consensus that there
> *are* two different problems to be solved here, and solving them both
> well in a single tool is likely to be nigh on impossible. (I'm aware
> we're really on the wrong list for that discussion, but I also think
> there's value in getting some broader python-dev awareness of this
> particular topic)

I'm not entirely sure what you're proposing here.

Obviously there are multiple classes of users (whether it's as simple
as just two, or whether there's more is less important).
Clearly there is a case for curated stacks and OS distributions, and
clearly some people use those stacks and distributions.
Things aren't perfect - CPython and projects like pip need to be aware
of, and respond to, the differing needs of people who use Python in
different ways.

But what are you suggesting python-dev needs to *do* about this? What
precisely is wrong with how we deal with the multiple types of user
that exist at the moment?

Without wanting to sound like I'm taking things personally, it feels
like there's an intention to suggest to *some* people that they would
be better served by a curated stack. I don't know how to answer that
except from a personal perspective[1], and it's hard to do that
without knowing whether I'm in a group that you'd see as benefiting
from a curated stack.

One thing I *would* suggest is that a lot of "corporate" use of Python
(by which I mean semi-informal scripting and automation done as part
of the infrastructure of larger projects written in more "enterprise"
tools like Java or higher level CRM/eBusiness/etc packages) is not
suitable for a curated stack (corporate IT policies would see that as
a "3rd party tool" where the python.org distribution is just a
project-internal utility). But the staff involved are typically not
familiar with open source or its culture, and struggle to deal with
things like package management (this is *not* the "legal approval"
issue, as cut and paste of code from websites is common in this
culture - it's "internal use only"). Within the context of your two
categories, this may well be a third one (unless you stretch
"application developers" way beyond what I think you are intending).

Paul

[1] By which I mean "looking at what I use Python for, how I see it
used by others in my organisation, and how I would expect to promote
Python to people who don't currently use it but whom I feel would
benefit from it".
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Donald Stufft]

On Apr 18, 2014, at 4:22 PM, Nick Coghlan  wrote:

> On 18 April 2014 15:39, Donald Stufft  wrote:
>> 
>> On Apr 18, 2014, at 3:18 PM, Nick Coghlan  wrote:
>> 
>>> At this point, however, I'm mainly looking for consensus that there
>>> *are* two different problems to be solved here, and solving them both
>>> well in a single tool is likely to be nigh on impossible. (I'm aware
>>> we're really on the wrong list for that discussion, but I also think
>>> there's value in getting some broader python-dev awareness of this
>>> particular topic)
>> 
>> I’m not sure about this? I mean yes those are two different areas, but I’m
>> not sure about the split between Conda and pip here. As far as I can tell
>> Conda is useful in the same way apt-get or yum is useful, you get a non
>> Python specific set of packages (because sometimes things aren’t pure
>> python) and you also remove a little bit of thinking about versions (although
>> honestly I think it’s possible to remove most of that for consumers of
>> packages).
> 
> You also get the ability to use that same system to update *Python
> itself*, regardless of platform. Being notified of and consuming
> CPython updates on Windows, as well as consuming alternate versions on
> Linux distros, is currently not a well solved problem - with conda,
> it's no more complex than updating a PyPI package.
> 
> That's one of the most attractive aspects for me - making Python 3.4,
> pypy, etc as easy to update with consistent cross platform
> instructions as Python packages are.

Ah right. I would never want that personally so I forgot about it. (I also 
don’t use
the system Python for development but I user a different tool for managing it).

> 
>> To be quite frank, a lot of the benefit of Conda outside of the “I need 
>> something
>> that isn’t strictly Python) is in the fact they can bootstrap compiled 
>> packages
>> whereas pip/wheel/PyPI combination we need to convince authors to upload
>> their own binary packages (or at least develop something to make it easier
>> like a build farm).
> 
> Yep, that's a large part of why I think "divide and conquer" is the
> way to go here. While it isn't completely accurate (as most SaaS
> developers don't want to build C extensions from source) I think
> Guido's "build from source" vs "install a pre-built binary"
> distinction is still a reasonably good way to characterise it. For a
> distro like Fedora (or, even more so, RHEL), we're not going to trust
> a binary created by someone else if we can possibly avoid it, so
> upstream binaries aren't useful to us, but pip's ability to abstract
> away the vagaries of the upcoming metadata 2.0 migration is incredibly
> helpful. The other distros are in the same situation (we'll always
> feed source tarballs into our own build systems), and ditto for the
> conda folks. We need that for all sorts of reasons that potential new
> Python users don't care about, but continuing to meet our
> requirements, along with the free-for-all that is PyPI makes handling
> the binary distribution problem *much* harder for pip.
> 
> By contrast, like any other distro, conda doesn't need to boil the
> ocean - it just needs to provide a filtered, up to date set of core
> packages that work well together. The advantage it has over other
> distros is that it is *cross-platform* - it works essentially the same
> way on Windows, Mac OS X. Most other package management systems are
> either platform specific or can't handle arbitrary binary
> dependencies. By being Python-centric (even if not Python specific),
> there's also a strong focus on updating the core packages more often
> than the Linux distros do.
> 
> There's no "always use conda instead of pip" competition here - we
> need pip, we need wheels. But I see those as software integrator
> focused tools that would need a lot of additional work to provide a
> truly spectacularly compelling out of the box experience for new
> users. I don't think that's a useful way to expend effort - I think it
> makes more sense to work on a separate "here's the fully assembled
> environment for using Python as a tool to develop ideas" introduction,
> while pushing the "here's how to build your own custom environment
> from upstream parts" as a later step in a new user's journey towards
> full Python mastery (if they're interested in that path). Making sure
> that "pip install foo" does the right thing inside conda environments
> (if it doesn't already) should be all that is needed to ensure that
> random installation instructions on the internet still work.

So I’m not really worried about a competition or anything. I’m mostly worried
about confusion of users. What you’re suggestion we give to use is *two* ways
to install Python packages (and 2 or 3 ways to virtualize a Python instance).
That provides extra cognitive burden for people who are new to Python. They
have to both understand that the packages you install from Conda are different
than the ones you install from pip, an

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Donald Stufft]

On Apr 18, 2014, at 4:50 PM, Donald Stufft  wrote:

> So I’m not really worried about a competition or anything. I’m mostly worried
> about confusion of users. What you’re suggestion we give to use is *two* ways
> to install Python packages (and 2 or 3 ways to virtualize a Python instance).
> That provides extra cognitive burden for people who are new to Python. They
> have to both understand that the packages you install from Conda are different
> than the ones you install from pip, and that they come from different places.
> If a new user is reasonably likely to have to use ``pip install`` when they 
> are
> using pip, then I think that provides a worse experience than only having to
> use one tool to manage your packages. This confusion is going to be worse when
> new users find a library they want to use and it tells them to use
> ``pip install`` (or even easy_install!) even if Anaconda itself has a package
> inside of it.

Typo - When they are using conda.
-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 16:27, Paul Moore  wrote:
> On 18 April 2014 20:18, Nick Coghlan  wrote:
>> At this point, however, I'm mainly looking for consensus that there
>> *are* two different problems to be solved here, and solving them both
>> well in a single tool is likely to be nigh on impossible. (I'm aware
>> we're really on the wrong list for that discussion, but I also think
>> there's value in getting some broader python-dev awareness of this
>> particular topic)
>
> I'm not entirely sure what you're proposing here.
>
> Obviously there are multiple classes of users (whether it's as simple
> as just two, or whether there's more is less important).
> Clearly there is a case for curated stacks and OS distributions, and
> clearly some people use those stacks and distributions.
> Things aren't perfect - CPython and projects like pip need to be aware
> of, and respond to, the differing needs of people who use Python in
> different ways.
>
> But what are you suggesting python-dev needs to *do* about this? What
> precisely is wrong with how we deal with the multiple types of user
> that exist at the moment?

When you go to python.org, you are currently advised to download one
of the following:

- a source tarball (Linux)
- a bare bones binary installer (Windows & Mac OS X)

What I am advocating for is that *we are currently doing it wrong*, as
these are unlikely to be the best thing to install for most new Python
users. Now, we could make things even *more* confusing by instead
presenting the prospective user with *even more* choices, like "oh,
you might want to get it from your Linux distro, or maybe homebrew, or
MacPorts", but that's not helping, because it's asking new users to
make choices that they may not be in a position to make.

>From a usability perspective, I don't believe we should be advertising
a minimalist installation as the preferred entry point for new users -
we should be advertising a full featured sumo distribution that lets
new users quickly experience the full power and flexibility of things
like IPython notebooks. I like Anaconda, because it's fully open
source, abstracts away the underlying operating system more than the
standard installers and the same mechanism that you use to update
packages can also be used to update the Python interpreter itself.

Everything else we do today should remain available for those that
want them - I'm not advocating taking anything away. But we need to
start getting more opinionated on behalf of new users that don't yet
have the experience they need to start challenging our judgement and
make their own choices about which tools to use out of the vast array
of choices the Python ecosystem provides.

> Without wanting to sound like I'm taking things personally, it feels
> like there's an intention to suggest to *some* people that they would
> be better served by a curated stack. I don't know how to answer that
> except from a personal perspective[1], and it's hard to do that
> without knowing whether I'm in a group that you'd see as benefiting
> from a curated stack.

No, I don't think you'd benefit substantially from a curated stack - I
don't think any experienced Python user would. But think through your
own personal stack. Start from a Google search for "Python tutorial"
and see how easily you can figure out how and where to obtain all
those components.

Putting a sumo distribution front and centre on the website
drastically improves the out of the box experience for new users,
*without* introducing a huge maintainability issue for the standard
library. New users gain quick access to a much larger fraction of the
overall Python ecosystem, while on the back end software *production*
side of things, everything stays decoupled.

It also provides a clearer timeline for adoption of new versions of
Python - while the reference interpreter would go up immediately, the
sumo version would only be updated once all the contained projects had
been checked for compatibility with the new version.

> One thing I *would* suggest is that a lot of "corporate" use of Python
> (by which I mean semi-informal scripting and automation done as part
> of the infrastructure of larger projects written in more "enterprise"
> tools like Java or higher level CRM/eBusiness/etc packages) is not
> suitable for a curated stack (corporate IT policies would see that as
> a "3rd party tool" where the python.org distribution is just a
> project-internal utility). But the staff involved are typically not
> familiar with open source or its culture, and struggle to deal with
> things like package management (this is *not* the "legal approval"
> issue, as cut and paste of code from websites is common in this
> culture - it's "internal use only"). Within the context of your two
> categories, this may well be a third one (unless you stretch
> "application developers" way beyond what I think you are intending).

No, that is the case covered by 'creators of corporate "Standard
Operating Environment" 

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 16:50, Donald Stufft  wrote:
> So I’m not really worried about a competition or anything. I’m mostly worried
> about confusion of users. What you’re suggestion we give to use is *two* ways
> to install Python packages (and 2 or 3 ways to virtualize a Python instance).

Note that one of my requirements was that "pip install foo" *must* do
the right thing in conda environments (whatever we decide the "right
thing" means in that context). It was buried at the end of a long
email though, so it would have been easy to miss.

That means the instructions to new users can be simple and consistent
- use pip commands to manage Python things, conda commands to manage
other stuff. They'll likely discover in fairly short order that the
conda commands also work for Python things, but it can be explained
that not all environments are conda environments, and hence pip works
in more situations than conda does, but at the cost of being specific
to Python packages.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] this is what happens if you freeze all the modules required for startup

[Ezio Melotti]

Hi,

On Thu, Apr 17, 2014 at 9:09 PM, Brett Cannon  wrote:
>
>
> On Thu Apr 17 2014 at 1:34:23 PM, Jurko Gospodnetić
>  wrote:
>>
>>Hi.
>>
>> On 14.4.2014. 23:51, Brett Cannon wrote:
>> > Now the question is whether the maintenance cost of having to rebuild
>> > Python for a select number of stdlib modules is enough to warrant
>> > putting in the effort to make this work.
>>
>>I would really love to have better startup times in production, but I
>> would also really hate to lose the ability to hack around in stdlib
>> sources during development just to get better startup performance.
>>
>>In general, what I really like about using Python for software
>> development is the ability to open any stdlib file and easily go poking
>> around using stuff like 'import pdb;pdb.set_trace()' or simple print
>> statements. Researching mysterious behaviour is generally much much
>> MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
>> into a stdlib Python module than if it takes you down into the bowels of
>> some C module (think zipimport.c *grin*). Not to mention the effect that
>> being able to quickly resolve a mystery by hacking on some Python
>> internals leaves you feeling very satisfied, while having to entrench
>> yourself in those internals for a long time just to find out you've made
>> something foolish on your end leaves you feeling exhausted at best.
>
>
> Freezing modules does not affect the ability to use gdb. And as long as you
> set the appropriate __file__ values then tracebacks will contain even the
> file line and location.
>

Will the tracebacks only contain the line number or the line of code too?

I've seen tracebacks involving importlib,_bootstrap that didn't
include the code, and I'm wondering if we will get something similar
for all the other modules you are freezing:

Traceback (most recent call last):
  File "/tmp/foo.py", line 7, in 
import email
  File "", line 1561, in _find_and_load
  File "", line 1519, in _find_and_load_unlocked
  File "", line 1473, in _find_module
  File "", line 1308, in find_module
  File "", line 1284, in _get_loader
  File "", line 1273, in _path_importer_cache
  File "", line 1254, in _path_hooks
TypeError: 'NoneType' object is not iterable

Best Regards,
Ezio Melotti



> -Brett
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ezio.melotti%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Paul Moore]

On 18 April 2014 21:59, Nick Coghlan  wrote:
> What I am advocating for is that *we are currently doing it wrong*, as
> these are unlikely to be the best thing to install for most new Python
> users.

For Windows users at least, I disagree. I have directed a lot of
people to the python.org Windows installers, and they have got up and
running without any issues (ignoring website usability "find the right
download" questions). Most of the time, they need nothing more than
the interpreter and the stdlib to become productive.

The downside to directing them at (say) conda is that 99.99% of the
documentation on the internet is not directed at conda users. Most of
the time, that's fine, because it's all generic stuff, but it's a trap
waiting to happen. I can't tell people to use conda myself as I'm not
a conda user, so I can't help them if they hit problems.

> No, I don't think you'd benefit substantially from a curated stack - I
> don't think any experienced Python user would. But think through your
> own personal stack. Start from a Google search for "Python tutorial"
> and see how easily you can figure out how and where to obtain all
> those components.

Core python, plus a standalone copy of virtualenv (and on 3.4+ I don't
even need that). Everything else is installed from PyPI as needed.
Agreed that starting from a Google search won't get that at the
moment, but that's what the packaging initiatives are trying to solve
- get the infrastructure there so that more and more people say the
same things, and so google searches become more accurate.

My vision of the ideal is:

1. Install Python from python.org
2. py -m venv foo
3. foo\Scripts\activate
4. pip install 

(Note that item 4 relies on pervasive availability of wheels for
packages like numpy, scipy, lxml, pywin32, etc)

Oh, and everyone needs to switch to Windows so this works :-)

More seriously, I genuinely think that this is the ideal solution for
*all* Windows users who don't have the sort of specialised needs that
mean a curated stack is right for them, and they know about it from
word of mouth via colleagues in the same situation.

I have no idea, and no opinion, on how practical it is to achieve this
level of simplicity on Unix or OSX. But I do think it's the right goal
to aim for for any cases where using "the system Python" is the wrong
answer.

>> One thing I *would* suggest is that a lot of "corporate" use of Python
>> (by which I mean semi-informal scripting and automation done as part
>> of the infrastructure of larger projects written in more "enterprise"
>> tools like Java or higher level CRM/eBusiness/etc packages) is not
>> suitable for a curated stack (corporate IT policies would see that as
>> a "3rd party tool" where the python.org distribution is just a
>> project-internal utility). But the staff involved are typically not
>> familiar with open source or its culture, and struggle to deal with
>> things like package management (this is *not* the "legal approval"
>> issue, as cut and paste of code from websites is common in this
>> culture - it's "internal use only"). Within the context of your two
>> categories, this may well be a third one (unless you stretch
>> "application developers" way beyond what I think you are intending).
>
> No, that is the case covered by 'creators of corporate "Standard
> Operating Environment" definitions'. That's explicitly in the software
> integrator category - whether those users formally have the
> responsibility of defining an SOE, that's what they're doing in
> practice.

OK. In which case that category is far broader than I anticipated, and
includes people who cannot even write a Windows batch file or use a
command line without instruction (I can name names!!! ;-)) If that's
your view, though, then all I can say is that I have no experience of
anyone who isn't in the "software integrator" category, so I'll take
your word that there is only one other category :-)

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Donald Stufft]

On Apr 18, 2014, at 5:08 PM, Nick Coghlan  wrote:

> On 18 April 2014 16:50, Donald Stufft  wrote:
>> So I’m not really worried about a competition or anything. I’m mostly worried
>> about confusion of users. What you’re suggestion we give to use is *two* ways
>> to install Python packages (and 2 or 3 ways to virtualize a Python instance).
> 
> Note that one of my requirements was that "pip install foo" *must* do
> the right thing in conda environments (whatever we decide the "right
> thing" means in that context). It was buried at the end of a long
> email though, so it would have been easy to miss.
> 
> That means the instructions to new users can be simple and consistent
> - use pip commands to manage Python things, conda commands to manage
> other stuff. They'll likely discover in fairly short order that the
> conda commands also work for Python things, but it can be explained
> that not all environments are conda environments, and hence pip works
> in more situations than conda does, but at the cost of being specific
> to Python packages.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia

Thinking about that and the implications of it.

Next question, where even is the code for “Anaconda”? I tried to download it
from their website and it’s behind an email form, I saw a link for github issues
but it’s in a dedicated “anaconda-issues” repo which doesn’t have any code
associated with it.

Also to be honest i’m a little uncomfortable with the idea of Python.org pushing
a platform where the company that develops the platform sells Add-ons to that
platform. So while Anaconda itself may be free and open source the fact that
the Anaconda distribution is a gateway to a particular company’s paid add ons
makes me feel a bit like a government sponsored monopoly kind of thing? I’m
not using good words here to describe what I mean, but it feels kind of icky to 
me.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Paul Moore]

On 18 April 2014 22:08, Nick Coghlan  wrote:
> Note that one of my requirements was that "pip install foo" *must* do
> the right thing in conda environments (whatever we decide the "right
> thing" means in that context).

Is this specifically a requirement for conda? Or do you expect the
same to be true for ActivePython, Python(X,Y) and whatever other
distributions exist? Assuming conda isn't a special case, is it down
to pip to ensure that or to the distributions to play nicely with pip?

It's starting to feel as if conda *is* being treated as somehow a
special case. (At least in the sense that you're advocating that we
promote a non-minimalist distribution on the python.org website, and
you've explicitly stated that you personally like conda). I'm strongly
against the official python.org website promoting *any* external
distribution as the recommended solution. I'm not against having a
simple list of "alternative distributions" if you think that'd help,
but that's a far cry from "advertising a full featured sumo
distribution that lets new users quickly experience the full power and
flexibility of things like IPython notebooks".

On the other hand, it might be good to have an explicit advertising
page - "See the power of Python" which *did* direct users to stacks
that showcase particularly impressive uses of Python. There could be a
section about the IPython notebook (which mentions that the conda
Python stack comes with IPython preinstalled) and another about Django
(hmm, what's the curated stack for web developers?) and another
showing the Visual Stiudio Tools for Python (hmm, stack for Windows
developers/admins?).

As a comparison, java.com points you straight at the standard Java SE
download. You don't see JBoss or Weblogic promoted on there, even
though those are likely to be how most developers interact with Java.
To get anything other than the minimal distribution, you need to know
you want it and search for it.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 17:22, Donald Stufft  wrote:
>
> On Apr 18, 2014, at 5:08 PM, Nick Coghlan  wrote:
>
>> On 18 April 2014 16:50, Donald Stufft  wrote:
>>> So I’m not really worried about a competition or anything. I’m mostly 
>>> worried
>>> about confusion of users. What you’re suggestion we give to use is *two* 
>>> ways
>>> to install Python packages (and 2 or 3 ways to virtualize a Python 
>>> instance).
>>
>> Note that one of my requirements was that "pip install foo" *must* do
>> the right thing in conda environments (whatever we decide the "right
>> thing" means in that context). It was buried at the end of a long
>> email though, so it would have been easy to miss.
>>
>> That means the instructions to new users can be simple and consistent
>> - use pip commands to manage Python things, conda commands to manage
>> other stuff. They'll likely discover in fairly short order that the
>> conda commands also work for Python things, but it can be explained
>> that not all environments are conda environments, and hence pip works
>> in more situations than conda does, but at the cost of being specific
>> to Python packages.
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
> Thinking about that and the implications of it.
>
> Next question, where even is the code for “Anaconda”? I tried to download it
> from their website and it’s behind an email form, I saw a link for github 
> issues
> but it’s in a dedicated “anaconda-issues” repo which doesn’t have any code
> associated with it.
>
> Also to be honest i’m a little uncomfortable with the idea of Python.org 
> pushing
> a platform where the company that develops the platform sells Add-ons to that
> platform. So while Anaconda itself may be free and open source the fact that
> the Anaconda distribution is a gateway to a particular company’s paid add ons
> makes me feel a bit like a government sponsored monopoly kind of thing? I’m
> not using good words here to describe what I mean, but it feels kind of icky 
> to me.

I actually agree and it wouldn't necessarily make sense for *Anaconda*
itself to be the promoted sumo distribution. I'm more interested in
the idea of making a sumo distribution the default recommended entry
point, and handling the issues of independent curation and
redistribution that would come with that, as well as the mechanics of
how to actually handle installation and updates from the end user
perspective.

Perhaps we can get the "pip install ipython" experience to a good
place faster than I currently expect, and we can duck this entire
question (at least for Windows and Mac OS X).

Regardless, I think we've drifted far enough into speculative
territory to be off-topic for python-dev. This isn't a question we're
going to resolve in a hurry, and the near term focus should likely be
on improving the numpy+wheel story, since that's valuable regardless
of what happens with the python.org download pages. While that is
ongoing, people can mull over the various possibilities independently.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] dict and required hashing

[Jim J. Jewett]

(1)  I believe the recent consensus was that the number of comparisons
made in a dict lookup is an implementation detail.  (Please correct me
if I am wrong.)

(2)  Is "the item will be hashed at least once" a language guarantee?

For small mappings, it might well be more efficient to just store the
2-3 key/value pairs and skip the bucket calculation.

On the other hand, if a key is not hashable, discovering that long
after it has already been added to the dict is suboptimal.

Of course, that sort of delayed exception can already happen if it is
the __eq__ method that is messed up ...

-jJ
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Paul Moore]

On 18 April 2014 22:40, Nick Coghlan  wrote:
> Perhaps we can get the "pip install ipython" experience to a good
> place faster than I currently expect, and we can duck this entire
> question (at least for Windows and Mac OS X).

Huh? Last time I tried, it was pretty trivial.

pip install pyzmq pyreadline tornado ipython

(I might have missed a couple, as I did this from memory, not by
looking at the website to check). There's nothing that needs a
compiler apart from pyzmq and that has wheels for Windows these days,
and builds out of the box with no dependencies in any case. The rest
is all pure Python.

I've not tried with 2.0, but I did try one of the betas and it was
really that simple. (Windows 7 64-bit, Python 3.4 IIRC). And the 1.x
versions were just as simple.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dict and required hashing

[Benjamin Peterson]

On Fri, Apr 18, 2014, at 14:46, Jim J. Jewett wrote:
> (1)  I believe the recent consensus was that the number of comparisons
> made in a dict lookup is an implementation detail.  (Please correct me
> if I am wrong.)

Absolutely.

> 
> (2)  Is "the item will be hashed at least once" a language guarantee?

No. (Would that be useful at all?)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Donald Stufft]

On Apr 18, 2014, at 5:48 PM, Paul Moore  wrote:

> On 18 April 2014 22:40, Nick Coghlan  wrote:
>> Perhaps we can get the "pip install ipython" experience to a good
>> place faster than I currently expect, and we can duck this entire
>> question (at least for Windows and Mac OS X).
> 
> Huh? Last time I tried, it was pretty trivial.
> 
> pip install pyzmq pyreadline tornado ipython
> 
> (I might have missed a couple, as I did this from memory, not by
> looking at the website to check). There's nothing that needs a
> compiler apart from pyzmq and that has wheels for Windows these days,
> and builds out of the box with no dependencies in any case. The rest
> is all pure Python.
> 
> I've not tried with 2.0, but I did try one of the betas and it was
> really that simple. (Windows 7 64-bit, Python 3.4 IIRC). And the 1.x
> versions were just as simple.
> 
> Paul

Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what 
that
includes. I’ve never used ipython except for the console.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Greg Ewing]

Nick Coghlan wrote:

there are actually now *two* main ways of consuming
Python:


Really? We'd better do something about that. We don't want
anyone consuming Python -- we want some left over for the
rest of us!

(I'm making a serious point -- it's annoying when people use
the word "consume" as though it were a synonym for "use".
It's not.)

When-I-consume-a-word-said-Humpty-Dumpty-ly,
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Paul Moore]

On 18 April 2014 22:57, Donald Stufft  wrote:
> Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what 
> that
> includes. I’ve never used ipython except for the console.

The hard bit is the QT Console, but that's because there aren't wheels
for PySide AFAICT.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 18:16, Greg Ewing  wrote:
> Nick Coghlan wrote:
>>
>> there are actually now *two* main ways of consuming
>> Python:
>
>
> Really? We'd better do something about that. We don't want
> anyone consuming Python -- we want some left over for the
> rest of us!
>
> (I'm making a serious point -- it's annoying when people use
> the word "consume" as though it were a synonym for "use".
> It's not.)

I don't consider "use" a synonym for "consume" in this case, as
consuming an upstream project covers redistribution as well. Now, I
could write "use or redistribute" everywhere, but "consume" is
shorter.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 18:17, Paul Moore  wrote:
> On 18 April 2014 22:57, Donald Stufft  wrote:
>> Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what 
>> that
>> includes. I’ve never used ipython except for the console.
>
> The hard bit is the QT Console, but that's because there aren't wheels
> for PySide AFAICT.

IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
let you break programming out of the low level box of controlling the
computer, and connect it directly to the more universal high level
task of understanding and visualising the world.

Regards,
Nick.

>
> Paul



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Donald Stufft]

On Apr 18, 2014, at 6:24 PM, Nick Coghlan  wrote:

> On 18 April 2014 18:17, Paul Moore  wrote:
>> On 18 April 2014 22:57, Donald Stufft  wrote:
>>> Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know 
>>> what that
>>> includes. I’ve never used ipython except for the console.
>> 
>> The hard bit is the QT Console, but that's because there aren't wheels
>> for PySide AFAICT.
> 
> IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
> let you break programming out of the low level box of controlling the
> computer, and connect it directly to the more universal high level
> task of understanding and visualising the world.
> 
> Regards,
> Nick.
> 
>> 
>> Paul
> 
> 
> 
> -- 
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia

FWIW It’s been David Cournapeau’s opinion (on Twitter at least) that 
some/all/most
(I’m not sure exactly which) of these can be handled by Wheels (they just 
aren’t right now!).

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Nick Coghlan]

On 18 April 2014 18:28, Donald Stufft  wrote:
>
> On Apr 18, 2014, at 6:24 PM, Nick Coghlan  wrote:
>
>> On 18 April 2014 18:17, Paul Moore  wrote:
>>> On 18 April 2014 22:57, Donald Stufft  wrote:
 Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know 
 what that
 includes. I’ve never used ipython except for the console.
>>>
>>> The hard bit is the QT Console, but that's because there aren't wheels
>>> for PySide AFAICT.
>>
>> IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
>> let you break programming out of the low level box of controlling the
>> computer, and connect it directly to the more universal high level
>> task of understanding and visualising the world.
>>
>> Regards,
>> Nick.
>>
>>>
>>> Paul
>>
>>
>>
>> --
>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
> FWIW It’s been David Cournapeau’s opinion (on Twitter at least) that 
> some/all/most
> (I’m not sure exactly which) of these can be handled by Wheels (they just 
> aren’t right now!).

Yeah, I think they're fixable too. And after thinking through the
implications of recommending a specific sumo distribution, that
actually does seem to be a more straightforward path as a "default
entry point".

I still see merit in working with the conda folks to make it easier
for Windows and Mac OS folks to keep their Python installations up to
date, and for Linux users to stay out of the system Python in a distro
independent manner, but that's a separate discussion from the
python.org download pages one.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Software integrators vs end users (was Re: Language Summit notes)

[Donald Stufft]

On Apr 18, 2014, at 6:37 PM, Nick Coghlan  wrote:

> On 18 April 2014 18:28, Donald Stufft  wrote:
>> 
>> On Apr 18, 2014, at 6:24 PM, Nick Coghlan  wrote:
>> 
>>> On 18 April 2014 18:17, Paul Moore  wrote:
 On 18 April 2014 22:57, Donald Stufft  wrote:
> Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know 
> what that
> includes. I’ve never used ipython except for the console.
 
 The hard bit is the QT Console, but that's because there aren't wheels
 for PySide AFAICT.
>>> 
>>> IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
>>> let you break programming out of the low level box of controlling the
>>> computer, and connect it directly to the more universal high level
>>> task of understanding and visualising the world.
>>> 
>>> Regards,
>>> Nick.
>>> 
 
 Paul
>>> 
>>> 
>>> 
>>> --
>>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>> 
>> FWIW It’s been David Cournapeau’s opinion (on Twitter at least) that 
>> some/all/most
>> (I’m not sure exactly which) of these can be handled by Wheels (they just 
>> aren’t right now!).
> 
> Yeah, I think they're fixable too. And after thinking through the
> implications of recommending a specific sumo distribution, that
> actually does seem to be a more straightforward path as a "default
> entry point".
> 
> I still see merit in working with the conda folks to make it easier
> for Windows and Mac OS folks to keep their Python installations up to
> date, and for Linux users to stay out of the system Python in a distro
> independent manner, but that's a separate discussion from the
> python.org download pages one.

Sure, and for *nix based ones there’s also pyenv which I personally use
and like :)

> 
> Cheers,
> Nick.
> 
> 
> -- 
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dict and required hashing

[Steven D'Aprano]

On Fri, Apr 18, 2014 at 02:57:55PM -0700, Benjamin Peterson wrote:
> On Fri, Apr 18, 2014, at 14:46, Jim J. Jewett wrote:
> > (1)  I believe the recent consensus was that the number of comparisons
> > made in a dict lookup is an implementation detail.  (Please correct me
> > if I am wrong.)
> 
> Absolutely.
> 
> > 
> > (2)  Is "the item will be hashed at least once" a language guarantee?
> 
> No. (Would that be useful at all?)

I'm curious under what circumstances a key won't be hashed at all.


-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dict and required hashing

[Benjamin Peterson]

On Fri, Apr 18, 2014, at 17:30, Steven D'Aprano wrote:
> On Fri, Apr 18, 2014 at 02:57:55PM -0700, Benjamin Peterson wrote:
> > On Fri, Apr 18, 2014, at 14:46, Jim J. Jewett wrote:
> > > (1)  I believe the recent consensus was that the number of comparisons
> > > made in a dict lookup is an implementation detail.  (Please correct me
> > > if I am wrong.)
> > 
> > Absolutely.
> > 
> > > 
> > > (2)  Is "the item will be hashed at least once" a language guarantee?
> > 
> > No. (Would that be useful at all?)
> 
> I'm curious under what circumstances a key won't be hashed at all.

It's probably always hashed now, but as Jim suggested, it might happen
in an optimization. For example, if you try to do a lookup on an empty
dict, you definitely don't need to hash anything.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] devguide: Add note about Kushal Das' privs

[Dr. Brett Cannon]

On Friday, April 18, 2014 3:10:54 PM, Zachary Ware <
zachary.ware+py...@gmail.com> wrote:

On Fri, Apr 18, 2014 at 1:54 PM, Brett Cannon 
gmail.com > wrote:
> On Friday, April 18, 2014 2:35:32 PM, Benjamin Peterson
> @ 
> python.org>
wrote:
>> On Fri, Apr 18, 2014, at 11:29, Antoine Pitrou wrote:
>> > On Mon, 14 Apr 2014 23:18:42 +0200 (CEST)
>> > brett.cannon > > checkins
@ python.org >
wrote:
>> > > http:// 
>> > > hg.python.org
/ 
devguide
/rev/c14c8a195fec 
>> > > changeset:   686:c14c8a195fec
>> > > user:Brett Cannon > > > python.org
>
>> > > date:Mon Apr 14 17:18:37 2014 -0400
>> > > summary:
>> > >   Add note about Kushal Das' privs
>> >
>> > I have no objection to Kushal getting commit rights, but is there a
>> > public record of the discussion leading to this decision? I can't find
>> > anything on either python-dev or python-committers (perhaps my search
>> > skills are failing me).
>>
>> Probably in person conversion at PyCon sprints
>
> What Benjamin said; discussions at PyCon amongst several of us.

Looking at that page, it doesn't look like I was ever added.  Brett,
was it you that enabled me



I don't remember. It's somewhat annoying keeping the log updated since it's
a separate repo from the ssh keys.

-brett


Also, welcome Kushal!

--
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Network security enhancements for Python 2.7.7

[Nick Coghlan]

On 18 April 2014 19:34, Guido van Rossum  wrote:
> Thanks, Nick. I hereby approve this PEP. You can update the status yourself.
> Congrats!

Thanks! PEP status updated: http://hg.python.org/peps/rev/d905b6f9c6a9

I also tweaked the type, since it's just an ordinary Standards Track
PEP now, rather than an ongoing policy PEP.

I filed separate tracker issues for the four parts of the proposal:

os.urandom: http://bugs.python.org/issue21305
hmac: http://bugs.python.org/issue21306
hashlib: http://bugs.python.org/issue21307
ssl:http://bugs.python.org/issue21308

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

[Nick Coghlan]

After spending some time talking to the folks at the PyCon Twisted
sprints, they persuaded me that adding back the iterkeys/values/items
methods for mapping objects would be a nice way to eliminate a key
porting hassle for them (and likely others), without significantly
increasing the complexity of Python 3.

I personally put this one in the same category as PEP 414 - not
particularly useful from a Python 3 perspective, but not really
harmful either, and helpful enough from a transition perspective to be
worth doing.

Regards,
Nick.

==
PEP: 469
Title: Simplified migration of iterator-based mapping code to Python 3
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-04-18
Python-Version: 3.5
Post-History: 2014-04-18


Abstract


For Python 3, PEP 3106 changed the design of the ``dict`` builtin and the
mapping API in general to replace the separate list based and iterator based
APIs in Python 2 with a merged, memory efficient set and multiset view
based API.

This means that Python 3 code always requires an additional qualifier to
reliably reproduce classic Python 2 mapping semantics:

* List based (e.g. ``d.keys()``): ``list(d.keys())``
* Iterator based (e.g. ``d.iterkeys()``): ``iter(d.keys())``

Some Python 2 code that uses ``d.keys()`` may be migrated to Python 3
(or the common subset of Python 2 and Python 3) without alteration, but
*all* code using the iterator based API requires modification. Code that
is migrating to the common subset of Python 2 and 3 and needs to retain the
memory efficient implementation that avoids creating an unnecessary list
object must switch away from using a method to instead using a helper
function (such as those provided by the ``six`` module)

To simplify the process of migrating Python 2 code that uses the existing
iterator based APIs to Python 3, this PEP proposes the reintroduction
of the Python 2 spelling of the iterator based semantics in Python 3.5, by
restoring the following methods to the builtin ``dict`` API and the
``collections.abc.Mapping`` ABC definition:

* ``iterkeys()``
* ``itervalues()``
* ``iteritems()``


Proposal


Methods with the following exact semantics will be added to the builtin
``dict`` type and ``collections.abc.Mapping`` ABC::

def iterkeys(self):
return iter(self.keys())

def itervalues(self):
return iter(self.values())

def iteritems(self):
return iter(self.items())

These semantics ensure that the methods also work as expected for subclasses
of these base types.


Rationale
=

Similar in spirit to PEP 414 (which restored explicit Unicode literal
support in Python 3.3), this PEP is aimed primarily at helping users
that currently feel punished for making use of a feature that needed to be
requested explicitly in Python 2, but was effectively made the default
behaviour in Python 3.

Users of list-based iteration in Python 2 that aren't actually relying on
those semantics get a free memory efficiency improvement when migrating to
Python 3, and face no additional difficulties when migrating via the common
subset of Python 2 and 3.

By contrast, users that actually want the increased efficiency may have
faced a three phase migration process by the time they have fully migrated
to Python 3:

* original migration to the iterator based APIs after they were added in
  Python 2.2
* migration to a separate function based API in order to run in the common
  subset of Python 2 and 3
* eventual migration back to unprefixed method APIs when finally dropping
  Python 2.7 support at some point in the future

The view based APIs that were added to Python 2.7 don't actually help with
the transition process, as they don't exist in Python 3 and hence aren't
part of the common subset of Python 2 and Python 3, and also aren't supported
by most Python 2 mappings (including the collection ABCs).

This PEP proposes to just eliminate all that annoyance by making the iterator
based APIs work again in Python 3.5+. As with the restoration of Unicode
literals, it does add a bit of additional noise to the definition of Python
3, but it does so while bringing a significant benefit in increasing the size
of the common subset of Python 2 and Python 3 and so simplifying the process
of migrating to Python 3 for affected Python 2 users.


Acknowledgements


Thanks to the folks at the Twisted sprint table at PyCon for a very
vigorous discussion of this idea (and several other topics), and especially
to Hynek Schlawack for acting as a moderator when things got a little too
heated :)


Copyright
=

This document has been placed in the public domain.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo

Re: [Python-Dev] dict and required hashing

[Victor Stinner]

Does it mean that depending of the number of items, keys can be mutable? It
sounds like a terrible idea.

Victor

Le vendredi 18 avril 2014, Jim J. Jewett  a écrit :

> (1)  I believe the recent consensus was that the number of comparisons
> made in a dict lookup is an implementation detail.  (Please correct me
> if I am wrong.)
>
> (2)  Is "the item will be hashed at least once" a language guarantee?
>
> For small mappings, it might well be more efficient to just store the
> 2-3 key/value pairs and skip the bucket calculation.
>
> On the other hand, if a key is not hashable, discovering that long
> after it has already been added to the dict is suboptimal.
>
> Of course, that sort of delayed exception can already happen if it is
> the __eq__ method that is messed up ...
>
> -jJ
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

[Benjamin Peterson]

On Fri, Apr 18, 2014, at 19:31, Nick Coghlan wrote:
> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.
> 
> I personally put this one in the same category as PEP 414 - not
> particularly useful from a Python 3 perspective, but not really
> harmful either, and helpful enough from a transition perspective to be
> worth doing.

It doesn't seem to be widely known that Python 2.7's dict has
viewkeys()/viewvalues()/viewitems() methods which implement Python 3
dictionary views. Thus, an alternate (or concurrent) proposal could be
add these aliases to the Python 3 dictionary type. At any rate, the PEP
should mention these methods' existence.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dict and required hashing

[Benjamin Peterson]

On Fri, Apr 18, 2014, at 20:05, Victor Stinner wrote:
> Does it mean that depending of the number of items, keys can be mutable?
> It
> sounds like a terrible idea.

I believe Jim is talking about internal implementation.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

[Steven D'Aprano]

On Fri, Apr 18, 2014 at 10:31:29PM -0400, Nick Coghlan wrote:
> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.

It would also considerable add to the cruft of Python 3. One motive for 
going through the pain of Python 2 to 3 migration was to remove cruft. 
Adding it back in again is not just an aid to porting but actively 
making Python 3 a worse (well, "less better") experience.

In the case of u'' syntax for strings, that was (somewhat) unavoidable, 
as you get a syntax error in Python 3 otherwise. So you cannot write:

if sys.version < "3":
s = u"êπд"
else:
s = "êπд"

it simply doesn't work. But you can write:

if sys.version < "3":
items = mydict.iteritems()  # or viewitems
else:
items = mydict.items()

Feature discovery is better than explicit version checks:

try:
items = mydict.iteritems()
except AttributeError:
items = mydict.items()


In my experience, writing polyglot 2+3 code can be easily handled with a 
few helper functions, which is not the case with unicode string 
literals. (Non-polygot code of course can just use the methods 
directly.) I don't see this is a problem to be solved, or even much of a 
nuisance. Polyglot code is never going to be quite as straightforward or 
idiomic as non-polyglot code, and that's a good thing, as it reminds the 
reader that they are dealing with polyglot code.

So while I'm sympathetic to wanting to ease the 2/3 transition, even at 
the expense of re-introducing some cruft when unavoidable, I don't think 
that the difficulty of dealing with dict items|iteritems|viewitems etc. 
justifies re-adding cruft.


[...] 
> Rationale
> =
> 
> Similar in spirit to PEP 414 (which restored explicit Unicode literal
> support in Python 3.3), this PEP is aimed primarily at helping users
> that currently feel punished for making use of a feature that needed to be
> requested explicitly in Python 2, but was effectively made the default
> behaviour in Python 3.

"Feel punished"? That's awfully strong language. It may even be true, in 
the sense that some people *feel* that they are being punished, but I 
think the barrier to doing something about that needs to be a bit 
higher, namely that they *actually are* being punished.

I think that if "write a helper function" is punishment, then nearly 
every programmer is being punished *all the time*. That's part of 
programming. And if every 2+3 helper is seen as punishment that needs to 
be reversed, then we'll end up with Python 3.5 or 3.6 being virtually 
the same as Python 2.7 only with a few extra features.


> Users of list-based iteration in Python 2 that aren't actually relying on
> those semantics get a free memory efficiency improvement when migrating to
> Python 3, and face no additional difficulties when migrating via the common
> subset of Python 2 and 3.
> 
> By contrast, users that actually want the increased efficiency may have
> faced a three phase migration process by the time they have fully migrated
> to Python 3:
> 
> * original migration to the iterator based APIs after they were added in
>   Python 2.2

That was a long time ago. Surely we're not counting new features 
introduced a decade ago as part of the cost of migrating to Python 3?


> * migration to a separate function based API in order to run in the common
>   subset of Python 2 and 3
> * eventual migration back to unprefixed method APIs when finally dropping
>   Python 2.7 support at some point in the future

I'm not actually seeing the problem here. Forget 2+3, what you've 
described is surely part of the process of dealing with nearly any 
multi-version code. Back when I was still writing code to support Python 
2.3 through 2.5, I had helper functions like this:

def sort(alist, key=None):
if key is None:
alist.sort()
else:
try:
alist.sort(key=key)
except TypeError:
# Python too old. Use the DSU idiom.
...

When I dropped support for 2.3, I eventually (but not at first) moved 
back from sort(mylist) to mylist.sort(). I don't see that the situation 
with iterkeys etc. is terribly different.


> This PEP proposes to just eliminate all that annoyance by making the iterator
> based APIs work again in Python 3.5+. As with the restoration of Unicode
> literals, it does add a bit of additional noise to the definition of Python
> 3, but it does so while bringing a significant benefit in increasing the size
> of the common subset of Python 2 and Python 3 and so simplifying the process
> of migrating to Python 3 for affected Python 2 users.

To me, this feels more like re-adding cruft for trivial benefit. I would 
want to see justification for why 

Re: [Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

[Terry Reedy]

On 4/18/2014 10:31 PM, Nick Coghlan wrote:

After spending some time talking to the folks at the PyCon Twisted
sprints, they persuaded me that adding back the iterkeys/values/items
methods for mapping objects would be a nice way to eliminate a key
porting hassle for them (and likely others), without significantly
increasing the complexity of Python 3.


I hate this idea. It strikes me as junking up Python3 with stuff it is 
well rid of. I think anything that can be left to the transition modules 
should be. The u'' syntax had to be in the language itself. This does 
not have to be.



I personally put this one in the same category as PEP 414 -


When I suggested that PEP 414 might be seen as a precedent for restoring 
more of Py2, I was trashed for saying so. "No, no, u'' is a unique case. 
[it is] This will be the last proposal like this." What will come next?


> not

particularly useful from a Python 3 perspective, but not really
harmful either,


It makes the language a bit harder to learn and remember and slightly 
more confusing.


It will not help inter-operating with Python before 3.5, at the earliest 
and cannot be backported. Most things in an independent module can be 
used with any 3.x.


I would have preferred that you started by presenting the problem on 
python-ideas with possible solutions, rather than as a finished PEP 
pushing my least favorite solution.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com