Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On Tue, Apr 14, 2015 at 08:55:07AM +0200, Piotr Ożarowski wrote: > > If you want python (which include /usr/bin/python), install it. If you > > want > > python3, then the interpreter you're looking for is found at > > /usr/bin/python3. > > > > There's no dilemma to solve. > > +1 +1, but only because I hate /usr/bin/python without a number. > PS will this meeting be streamed or recording available somewhere later? Sadly no, this will just be a few of us during the sprints -- I'm planning on playing notekeeper, and I'll publish a summary of what was discussed to this thread, if that works. Paul -- .''`. Paul Tagliamonte | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt signature.asc Description: Digital signature
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
> I'm > planning on playing notekeeper, and I'll publish a summary of what was > discussed to this thread, if that works. great, thanks -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150414115737.gg2...@sar0.p1otr.com
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On Apr 14, 2015, at 12:38 AM, Scott Kitterman wrote: >If you want python (which include /usr/bin/python), install it. If you want >python3, then the interpreter you're looking for is found at /usr/bin/python3. I just don't want it to fail mysteriously. When there's no Python 2 by default, command-not-found will solve the type-it-at-the-shell failure mode just fine: $ python The program 'python is currently not installed. You can install it by typing: sudo apt-get install python-for-dinosaurs But it fails unhelpfully when you use it in a shebang. $ /tmp/foo.py bash: /tmp/foo.py: /usr/bin/python: bad interpreter: No such file or directory Let's make the latter more helpful. Cheers, -Barry pgp6w4t1eCOgX.pgp Description: OpenPGP digital signature
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On 14 April 2015 at 08:10, Barry Warsaw wrote: > But it fails unhelpfully when you use it in a shebang. > > $ /tmp/foo.py > bash: /tmp/foo.py: /usr/bin/python: bad interpreter: No such file or > directory > > Let's make the latter more helpful. > >From a script authors point of view, it's currently safe to assume that a shebang like '#!/usr/bin/env python' will work on any Linux machine. In some cases (Arch) it may already refer to Python 3, but with some care it's entirely possible to write a script that can do the right thing on Python 2 or 3. If distros start to remove 'python', there's an interim period before it's safe to assume that 'python3' is available everywhere, and script authors currently don't have any good options to bridge that. I know Debian is all in on treating Python 2 and 3 as two entirely separate worlds, but that's not how everyone sees them. It would be nice to make some kind of affordance to people for whom they are two versions of the same language. Thomas
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On Tuesday, April 14, 2015 08:10:49 AM Barry Warsaw wrote: > On Apr 14, 2015, at 12:38 AM, Scott Kitterman wrote: > >If you want python (which include /usr/bin/python), install it. If you > >want python3, then the interpreter you're looking for is found at > >/usr/bin/python3. > I just don't want it to fail mysteriously. > > When there's no Python 2 by default, command-not-found will solve the > type-it-at-the-shell failure mode just fine: > > $ python > The program 'python is currently not installed. You can install it by > typing: sudo apt-get install python-for-dinosaurs > > But it fails unhelpfully when you use it in a shebang. > > $ /tmp/foo.py > bash: /tmp/foo.py: /usr/bin/python: bad interpreter: No such file or > directory > > Let's make the latter more helpful. OK, but running an incompatible interpreter doesn't fall into that category. Scott K signature.asc Description: This is a digitally signed message part.
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On Tuesday, April 14, 2015 08:24:01 AM Thomas Kluyver wrote: > On 14 April 2015 at 08:10, Barry Warsaw wrote: > > But it fails unhelpfully when you use it in a shebang. > > > > $ /tmp/foo.py > > bash: /tmp/foo.py: /usr/bin/python: bad interpreter: No such file or > > directory > > > > Let's make the latter more helpful. > > > From a script authors point of view, it's currently safe to assume that a > shebang like '#!/usr/bin/env python' will work on any Linux machine. In > some cases (Arch) it may already refer to Python 3, but with some care it's > entirely possible to write a script that can do the right thing on Python 2 > or 3. If distros start to remove 'python', there's an interim period before > it's safe to assume that 'python3' is available everywhere, and script > authors currently don't have any good options to bridge that. > > I know Debian is all in on treating Python 2 and 3 as two entirely separate > worlds, but that's not how everyone sees them. It would be nice to make > some kind of affordance to people for whom they are two versions of the > same language. My concern regarding the future of /usr/bin/python isn't for things that are being updated, but for things that aren't. Anything written for python3 already uses /usr/bin/python3, so there's no forward compatibility issue. I have scripts I use locally that are untouched in almost a decade that use /usr/bin/python. They work and require no maintenance. I've not ported them to python3, because there's no need. It would seriously break my expectations as an admin if at some point I upgraded to a new version of Debian and /usr/bin/python magically switched to python3 and all my stuff broke. It's at least half a decade too soon to even think about this (I know Arch went insane, but that's their problem IMO). Scott K -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/9342632.b15tYHvqKN@kitterma-e6430
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On 14 April 2015 at 08:57, Scott Kitterman wrote: > I have scripts I use locally that are untouched in almost a decade that use > /usr/bin/python. I'm thinking about scripts that are written and distributed to people running on different, unknown, Linux distros. Obviously if you're only targeting your own machines, there's no problem. But if you want to write a script that will work for arbitrary Linux users, what should you do? I imagine it's not yet a safe assumption that python3 is installed everywhere, but on the other hand, Ubuntu and Fedora are both looking at dropping Python 2, so without some kind of compatibility shim it won't be safe to assume there's a python command. Thomas
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On Tuesday, April 14, 2015 09:22:22 AM Thomas Kluyver wrote: > On 14 April 2015 at 08:57, Scott Kitterman wrote: > > I have scripts I use locally that are untouched in almost a decade that > > use > > /usr/bin/python. > > I'm thinking about scripts that are written and distributed to people > running on different, unknown, Linux distros. Obviously if you're only > targeting your own machines, there's no problem. But if you want to write a > script that will work for arbitrary Linux users, what should you do? I > imagine it's not yet a safe assumption that python3 is installed > everywhere, but on the other hand, Ubuntu and Fedora are both looking at > dropping Python 2, so without some kind of compatibility shim it won't be > safe to assume there's a python command. I'm using myself as a stand in for users generally in this example. I'm sure I'm not alone in this. Making /usr/bin/python point at python3 is just bad design. It is, as I think it was you said, easy enough to write Python code these days that works for both python and python3. As an upstream developer, go ahead and do that and leave it to the distros to packageit appropriately for their environment. In the case of Debian/Ubuntu, if something needs python (vice python3), it should have proper dependencies declared. What's there or not by default doesn't matter. I don't think it's possible to have something that at runtime call python2.7 or python3 based on what's in the code. The only thing it could select based on is what's in the shebang and that helps not at all because if the shebang isn't /usr/bin/python the launcher won't get called and if it is, you have to assume it's python2.7 that's meant. Scott K -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/2559043.CTEsoOA8ER@kitterma-e6430
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On 2015-04-14 at 09:22:22 -0400, Thomas Kluyver wrote: > I'm thinking about scripts that are written and distributed to people > running on different, unknown, Linux distros. Obviously if you're only > targeting your own machines, there's no problem. But if you want to write a > script that will work for arbitrary Linux users, what should you do? I > imagine it's not yet a safe assumption that python3 is installed > everywhere, but on the other hand, Ubuntu and Fedora are both looking at > dropping Python 2, so without some kind of compatibility shim it won't be > safe to assume there's a python command. If I was writing something new today I would use python3, put python3 in the shebang and just say that it needs python3 as a dependency, and let the users install it from the package manager if they need it. The only machines I can think of where python3 may not be available even as a package are old redhat ones where the version of python2 is so ancient that it doesn't have the compatibility features required to run py2/py3 code anyway. -- Elena ``of Valhalla'' -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150414140911.gc9...@virginsteele.home.trueelena.org
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
On Tue, Apr 14, 2015 at 10:00:07AM -0400, Scott Kitterman wrote: > It is, as I think it was you said, easy enough to write Python code these > days > that works for both python and python3. As an upstream developer, go ahead > and do that and leave it to the distros to packageit appropriately for their > environment. FWIW; the only place I even touch Python 2 code is in Debian, 100% of work and personal software is Python 3. So, I'm a bit of a radical here, but I'm pretty anti-Python 2. Most of my projects don't support Python 2 at all. > I don't think it's possible to have something that at runtime call python2.7 > or python3 based on what's in the code. I believe the shim would be to yell at the user to install a legacy thing, not switch between the two, that was a joke Barry made during the Language summit. Howabout we wait until *after* we try to hash this out in person. I'll write up thoughts and we can form consensus after, on list and in a public way. I think the spread of opinions voiced in this thread will be well represented in the room, so it should go well to figure out stuff that would be practical and implementable.. If anyone has *different* oppinions, you should give me an off-thread ping with your thoughts so I can bring them up today. Cheers, Tag -- .''`. Paul Tagliamonte | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt signature.asc Description: Digital signature
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
I'm trying to make sure I understand this subthread correctly. Is the following an accurate summary of the desires? 1. It should be possible, in Debian, not to ship Python 2 by default in the near future, and to remove Python 2 from the archive in the far future. 2. A huge number of existing/"legacy" scripts use #!/usr/bin/env python or perhaps #!/usr/bin/python to mean Python 2. Until at least 2020 (when Python 2.7 is desupported), it should be possible to install Python 2.7, whether through the package manager or otherwise, and have these scripts run correctly. If this is done, it should not break unrelated things in Debian, like Python 3.x-only scripts that ship with Debian. 3. It should be possible for authors of new both-Python-2-and-3 source to write scripts so that they run on both an existing/"legacy" machine with Python 2.x installed only, and on future machines with Python 3.x installed only. "Legacy" machines will have Python 2 at /usr/bin/python, and no wrapper. Future machines might have a wrapper. (For instance, it's OK if the mechanism to solve this does not address current Ubuntu releases that ship Python 3 only by default.) I think this is solvable by defining a standard to identify polyglot Python 2/3 source. For instance, say that polyglot scripts should start #!/usr/bin/env python # py23 and add a wrapper in /usr/bin (perhaps via dpkg-divert) that checks to see if the second line of its input is "# py23" (or /^# *py23$/, or something). If so, it execs Python 3; if not, and if it can't find Python 2, it prints a useful error to stderr and exits. (For interactive use, it could print a warning and run Python 3, or silently exec Python 3, or something.) -- Geoffrey Thomas https://ldpreload.com geo...@ldpreload.com -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/alpine.deb.2.10.1504141028400.18...@cactuar.ldpreload.com
Re: PyCon BoF: Stretch goals for cPython, PyPy & CFFI
Hi debian-python (2015.04.13_22:17:03_+0200) > Matthias and I are planning to have a Debian Python BoF at PyCon, > tomorrow afternoon. I think lunch is 2pm, so 3pm? > > Meet outside the cPython sprint room? In case you didn't see the private mails: We've got a table in room 513b. Still on for 3pm SR -- Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150414182725.ga3...@bach.rivera.co.za
Bits from the Debian PyCon Hangout
=== BITS FROM THE DEBIAN PYCON HANGOUT ===
Agenda:
- Discuss how we might support multiple interpreters with Python 3
packages, for cpython + pypy C extensions.
- Set up a plan for the svn => git migration
- Python 2 "deprecation"
- /usr/bin/python - what do?
CPYTHON + PYPY EXTENSIONS
-
[APP]<-- pypy Application
\
[python-foobar]<-- Python extension
\
[python-barbaz] <-- C extension
The issue is, how do we provide a module [python-barbaz], such that this
extension will work for application [APP], when [APP] is running under PyPy.
Since [APP] only (really) depends on [python-foobar], dragging the dependency
of something like [pypy-barbaz] to [APP]'s control is sub-optimal.
The first pass solution is to ship *both* cpython and pypy extensions in
[python-barbaz], but don't add a Dependency on `pypy`.
Another solution was to create a helper that crawls the depdencies of [APP],
and pull in Depends, on a package like [pypy-barbaz], and use those.
Consensus seems to be "give it a shot" and try to see what works.
There are no pypy apps, so this isn't an issue yet.
SVN => GIT
--
"We should just do it!"
We've settled on a deadline of Jessie release (yes, like 2 weeks!) to convert
the majority of packages *or* have a *very* clear plan. We need to also
make a patch system choice, and the deadline is the same.
The current primary patch systems for git are git-dpm and patch-queue. We'd
likely have to pick one of the two.
All present felt strongly that we should always use pristine upstream tarballs
as released by upstreams, with pristine-tar.
PYTHON 2 "DEPRECATION"
--
Python 2 is EOL in 2020. Realistically, this means we have 2 cycles left to
get our packages off Python 2. This means we need to start planning *now* to
gut as much Python 2 from the archive as we can.
The end-goal is to get Python 2 off the default install ASAP, and make it so
hard to drag Python 2 in as a Dependency that you have to explicitly apt-get
install python 2.
- Extend lintian to raise warnings if a package only supports Python 2.
!! We need a contributor for this.
- Research all packages whose upstream supports Python 3, but we only build
Python 2 modules. File higher priority bugs to enable Python 3.
- File wishlist bugs on things that only use Python 2. We'll use usertags
to track progress.
- Find applications which work in Python 3, and aggresively move them to
Python 3, ensure Dependency chains are complete.
- Look at Python 2 modules which contain no r-deps, mark them as canidates
for removal. In 2 cycles time.
- Look at packages that are Python 2 only and are dead upstream or have no
support for Python 3 upstream. Patch packages to support alternatives,
and remove the upstream dead Python 2 stuff before stretch.
- Require new uploads supporting Python 3 upstream, to provide Python 3
packages.
Next, we'll need to research what Debian infra currently uses Python 2. This
is going to be huge. We need to start to get the Debian Infra off Python 2 and
onto Python 3. There are some massive projects here, so this one might
be sticky.
/usr/bin/python
===
"This will be contentious"
Upstream Python's direction for Python paths is in favor of explicitly numbered
/usr/bin/python2 and /usr/bin/python3. In support of this, rough consensus in
the room is that /usr/bin/python should likely be removed *entirely* from
shebangs (though not from the distro).
/usr/bin/python2 exists as far back as wheezy, so there's no need to try to
fiddle around, even for oldoldstable backports.
Plans:
- Explicitly define the shebang as being one of {python2,python3}
- Add a lintian warning for files shipping /usr/bin/python.
geofft's suggestion was considered, consensus was to do such work upstream,
and formalize it as a PEP (or update an existing PEP).
Many thanks to Allison Randal, Asheesh Laroia, Barry Warsaw, Geoffrey Thomas,
Matthias Klose, and Stefano Rivera (I think that's everyone? Sorry if I missed
anyone!)
With love,
Paul
--
.''`. Paul Tagliamonte | Proud Debian Developer
: :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87
`. `'` http://people.debian.org/~paultag
`- http://people.debian.org/~paultag/conduct-statement.txt
signature.asc
Description: Digital signature
Re: Bits from the Debian PyCon Hangout
Paul Tagliamonte writes: > === BITS FROM THE DEBIAN PYCON HANGOUT === Thanks for posting this so promptly, Paul! -- \ “Not using Microsoft products is like being a non-smoker 40 or | `\ 50 years ago: You can choose not to smoke, yourself, but it's | _o__) hard to avoid second-hand smoke.” —Michael Tiemann | Ben Finney -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/85egnmuxh1@benfinney.id.au
Upstream source merge only when building Debian source (was: Bits from the Debian PyCon Hangout)
Paul Tagliamonte writes: > SVN => GIT > -- > "We should just do it!" +1 […] > All present felt strongly that we should always use pristine upstream > tarballs as released by upstreams, with pristine-tar. I'm glad of the former. I don't use ‘pristine-tar’, though. I use the “merge when building the source package” workflow, where the upstream source is a tarball outside the working tree, not part of the Debian packaging VCS at all. See ‘git-buildpackage(1)’, the ‘--git-overlay’ option. Is that still a wholly compatible workflow with what is being proposed? -- \ “Admiration, n. Our polite recognition of another's resemblance | `\to ourselves.” —Ambrose Bierce, _The Devil's Dictionary_, 1906 | _o__) | Ben Finney -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/858uduux99@benfinney.id.au
Re: Bits from the Debian PyCon Hangout
On April 14, 2015 6:01:56 PM EDT, Paul Tagliamonte wrote:
>
>=== BITS FROM THE DEBIAN PYCON HANGOUT ===
>
>
>Agenda:
>
> - Discuss how we might support multiple interpreters with Python 3
>packages, for cpython + pypy C extensions.
> - Set up a plan for the svn => git migration
> - Python 2 "deprecation"
> - /usr/bin/python - what do?
>
>
>
>CPYTHON + PYPY EXTENSIONS
>-
>
> [APP]<-- pypy Application
> \
> [python-foobar]<-- Python extension
>\
> [python-barbaz] <-- C extension
>
>
>The issue is, how do we provide a module [python-barbaz], such that
>this
>extension will work for application [APP], when [APP] is running under
>PyPy.
>Since [APP] only (really) depends on [python-foobar], dragging the
>dependency
>of something like [pypy-barbaz] to [APP]'s control is sub-optimal.
>
>The first pass solution is to ship *both* cpython and pypy extensions
>in
>[python-barbaz], but don't add a Dependency on `pypy`.
>
>Another solution was to create a helper that crawls the depdencies of
>[APP],
>and pull in Depends, on a package like [pypy-barbaz], and use those.
>
>Consensus seems to be "give it a shot" and try to see what works.
>There are no pypy apps, so this isn't an issue yet.
What is the "it" that's to be given a shot? I see two choices there? Did you
discuss the package size implications of embedding the pypy extension in the
python-foo binary?
>SVN => GIT
>--
>
>"We should just do it!"
>
>We've settled on a deadline of Jessie release (yes, like 2 weeks!) to
>convert
>the majority of packages *or* have a *very* clear plan. We need to also
>make a patch system choice, and the deadline is the same.
>
>The current primary patch systems for git are git-dpm and patch-queue.
>We'd
>likely have to pick one of the two.
>
>All present felt strongly that we should always use pristine upstream
>tarballs
>as released by upstreams, with pristine-tar.
>
>
>PYTHON 2 "DEPRECATION"
>--
>
>Python 2 is EOL in 2020. Realistically, this means we have 2 cycles
>left to
>get our packages off Python 2. This means we need to start planning
>*now* to
>gut as much Python 2 from the archive as we can.
>
>The end-goal is to get Python 2 off the default install ASAP, and make
>it so
>hard to drag Python 2 in as a Dependency that you have to explicitly
>apt-get
>install python 2.
>
>- Extend lintian to raise warnings if a package only supports Python 2.
>!! We need a contributor for this.
>
>- Research all packages whose upstream supports Python 3, but we only
>build
> Python 2 modules. File higher priority bugs to enable Python 3.
>
>- File wishlist bugs on things that only use Python 2. We'll use
>usertags
> to track progress.
>
>- Find applications which work in Python 3, and aggresively move them
>to
> Python 3, ensure Dependency chains are complete.
>
>- Look at Python 2 modules which contain no r-deps, mark them as
>canidates
> for removal. In 2 cycles time.
>
>- Look at packages that are Python 2 only and are dead upstream or have
>no
> support for Python 3 upstream. Patch packages to support alternatives,
> and remove the upstream dead Python 2 stuff before stretch.
>
>- Require new uploads supporting Python 3 upstream, to provide Python 3
> packages.
>
>Next, we'll need to research what Debian infra currently uses Python 2.
>This
>is going to be huge. We need to start to get the Debian Infra off
>Python 2 and
>onto Python 3. There are some massive projects here, so this one might
>be sticky.
>
>
>/usr/bin/python
>===
>
>"This will be contentious"
I'm more confused than angry ...
>Upstream Python's direction for Python paths is in favor of explicitly
>numbered
>/usr/bin/python2 and /usr/bin/python3. In support of this, rough
>consensus in
>the room is that /usr/bin/python should likely be removed *entirely*
>from
>shebangs (though not from the distro).
Unless we are also removing /usr/bin/python, why does this matter? Unless there
will be cases where /usr/bin/python2 exists while /usr/bin/python doesn't, I
don't see what this does?
>/usr/bin/python2 exists as far back as wheezy, so there's no need to
>try to
>fiddle around, even for oldoldstable backports.
>
>Plans:
>
>- Explicitly define the shebang as being one of {python2,python3}
>- Add a lintian warning for files shipping /usr/bin/python.
As above, what's the benefit of this.
>
>geofft's suggestion was considered, consensus was to do such work
>upstream,
>and formalize it as a PEP (or update an existing PEP).
>
I think formalizing the eventual demise of /usr/bin/python would be a good
idea.
>
>Many thanks to Allison Randal, Asheesh Laroia, Barry Warsaw, Geoffrey
>Thomas,
>Matthias Klose, and Stefano Rivera (I think that's everyone? Sorry if I
>missed
>anyone!)
Yes, thanks.
Scott K
--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a su
