Re: Python 3.4 and ensurepip (rehashed, long)
- Original Message - > On Mar 21, 2014, at 04:30 AM, Bohuslav Kabrda wrote: > I'd still like something like that. On Debian, Python adds a /usr/local > directory to sys.path, but it's not /usr/local/lib/pythonX.Y/site-packages, > it's .../dist-packages for reasons we've long hashed out and I don't want to > re-open. Suffice to say that we do have a directory where `sudo pip install` > could install to that would play nice with system Python but wouldn't > interfere with distro packages. While I don't much like `sudo pip install` > as > a general recommendation, in principle we support that. So it seems that we should really concentrate this discussion. On Fedora's python-devel, Nick said that he'd like to hack up adding more load paths at PyCon next month and in this thread Donald said that he has an idea for PEP for that. We, as Fedora Python and pip maintainers would love to see something like that and Debian already has a version of it downstream. This really starts to look like we should all join one mailing list and discuss this in one thread :) What about the pypa ML? I guess joining one more ML is worth this... > Just to keep in mind when proposing this upstream that distros will have > similar goals, but different concrete paths. > > >> What will rewheeling prefer when: > >> > >> * bundled ones are newer than system ones? > > > >That's not currently implemented in our patch, since we'll be removing > >bundled wheels and relying on system versions only. But to solve this > >situation for the upstream patch, we'll do it like this > >- if user runs just "python3 -m ensurepip", nothing will happen > >- if user runs "python3 -m ensurepip --upgrade", his setuptools/pip will get > >upgraded > >Note, that this corresponds with current ensurepip documentation [2] of such > >situations. > > I read the patch, and I think I get it, but just to be clear, rewheeling > only covers pip and setuptools, not arbitrary other packages, right? rewheel should be able to create a wheel from arbitrary package, although there are some known issues that we're still trying to figure out. It is true that we'd also need to explictly rewheel all pip's dependencies in the patch for ensurepip and that part of the patch would remain downstream only, since upstream would use bundled pip with bundled requests etc. > Cheers, > -Barry -- Regards, Slavek. -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/41201529.3901953.1395821567294.javamail.zim...@redhat.com
Re: Python 3.4 and ensurepip (rehashed, long)
[Robert Collins, 2014-03-26] > On 26 March 2014 17:15, Scott Kitterman wrote: > > On Tuesday, March 25, 2014 15:29:06 Barry Warsaw wrote: > >> On Mar 25, 2014, at 03:19 PM, Donald Stufft wrote: > >> >I assume once someone has installed pip with apt-get they'd still be able > >> >to run pip install --upgrade pip if they wanted too? > >> > >> I would think they should be able to do that. > > > > If I've install a package and it's upgraded (this is for the system, not for > > any kind of virtualized/isolated environment), I would find it quite > > surprising > > and unfortunate that it upgraded itself from an external source. > > I'd be very surprised if a package manager told to upgrade itself used > a different source for its own code vs things it manages. > > Yes, people that use pip to install things globally deserve to keep > both pieces, but either prohibit it entirely, or have it work as > advertised, not some frankenstein. let¹ `sudo pip install ...` (or `python3.X -m ensurepip ...` ) install eggs/whls into /usr/local/pythonX.Y/dist-packages and `sudo pip upgrade` (or `python3.Y -m ensurepip --upgrade`) upgrade *only* eggs/whls from this directory. This way system packages are not touched and it's easy to clean after pip (`find /usr/local/pythonX.Y/dist-packages/ -name '*.(egg|whl) -delete') or copy eggs/whls to virtualenv. [¹] if used with --i-will-not-blame-debian or i-will-not-blame-debian=true in ~/.pydistutils.cfg -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140326130056.gc30...@sts0.p1otr.com
Re: Python 3.4 and ensurepip (rehashed, long)
[Barry Warsaw, 2014-03-25] > One of the things I'd like to see, in addition to supporting > platform-specified system-level installation directories > (i.e. /usr/local/lib/pythonX.Y/dist-packages on Debian), is an upstream switch > to tell Python to ignore this directory, mirroring e.g. -s. That way, if > people say "Well I just pip installed foo into /usr/local and it broke my > system" we'd be able to respond "you better use pythonX.Y --dont-blame-us". I like the idea (where --dont-blame-us simply disables support for .egg/.zip/.whl in the interpreter, assuming pip/easy_install installs zips instead of unpacking them) -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140326131210.gd30...@sts0.p1otr.com
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 26, 2014, at 9:12 AM, Piotr Ożarowski wrote: > [Barry Warsaw, 2014-03-25] >> One of the things I'd like to see, in addition to supporting >> platform-specified system-level installation directories >> (i.e. /usr/local/lib/pythonX.Y/dist-packages on Debian), is an upstream >> switch >> to tell Python to ignore this directory, mirroring e.g. -s. That way, if >> people say "Well I just pip installed foo into /usr/local and it broke my >> system" we'd be able to respond "you better use pythonX.Y --dont-blame-us". > > I like the idea (where --dont-blame-us simply disables support for > .egg/.zip/.whl in the interpreter, assuming pip/easy_install installs > zips instead of unpacking them) > -- > Piotr Ożarowski Debian GNU/Linux Developer > www.ozarowski.pl www.griffith.cc www.debian.org > GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 > > > -- > To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org > with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org > Archive: https://lists.debian.org/20140326131210.gd30...@sts0.p1otr.com > Pip does not install as an .egg or a .whl. It doesn’t use .egg at all and regardless of which format it downloads it unpacks it and installs it. There is no difference in format between what pip installs and any debian package I’ve ever looked at. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Python 3.4 and ensurepip (rehashed, long)
[Donald Stufft, 2014-03-26] > On Mar 26, 2014, at 9:12 AM, Piotr Ożarowski wrote: > > [Barry Warsaw, 2014-03-25] > >> One of the things I'd like to see, in addition to supporting > >> platform-specified system-level installation directories > >> (i.e. /usr/local/lib/pythonX.Y/dist-packages on Debian), is an upstream > >> switch > >> to tell Python to ignore this directory, mirroring e.g. -s. That way, if > >> people say "Well I just pip installed foo into /usr/local and it broke my > >> system" we'd be able to respond "you better use pythonX.Y --dont-blame-us". > > > > I like the idea (where --dont-blame-us simply disables support for > > .egg/.zip/.whl in the interpreter, assuming pip/easy_install installs > > zips instead of unpacking them) > > Pip does not install as an .egg or a .whl. It doesn’t use .egg at all and > regardless > of which format it downloads it unpacks it and installs it. There is no > difference in > format between what pip installs and any debian package I’ve ever looked at. then --dont-blame-us has to mean removing /usr/local from sys.path -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140326131639.ge30...@sts0.p1otr.com
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 26, 2014, at 9:16 AM, Piotr Ożarowski wrote: > [Donald Stufft, 2014-03-26] >> On Mar 26, 2014, at 9:12 AM, Piotr Ożarowski wrote: >>> [Barry Warsaw, 2014-03-25] One of the things I'd like to see, in addition to supporting platform-specified system-level installation directories (i.e. /usr/local/lib/pythonX.Y/dist-packages on Debian), is an upstream switch to tell Python to ignore this directory, mirroring e.g. -s. That way, if people say "Well I just pip installed foo into /usr/local and it broke my system" we'd be able to respond "you better use pythonX.Y --dont-blame-us". >>> >>> I like the idea (where --dont-blame-us simply disables support for >>> .egg/.zip/.whl in the interpreter, assuming pip/easy_install installs >>> zips instead of unpacking them) >> >> Pip does not install as an .egg or a .whl. It doesn’t use .egg at all and >> regardless >> of which format it downloads it unpacks it and installs it. There is no >> difference in >> format between what pip installs and any debian package I’ve ever looked at. > > then --dont-blame-us has to mean removing /usr/local from sys.path > -- > Piotr Ożarowski Debian GNU/Linux Developer > www.ozarowski.pl www.griffith.cc www.debian.org > GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 > > > -- > To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org > with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org > Archive: https://lists.debian.org/20140326131639.ge30...@sts0.p1otr.com > In my half formed idea in my head the way it’d work is there’d be a vendor-packages directory where downstream can install things to, and a flag to the interpreter to remove the typical site-packages. So then you’d get something like: python -I —no-site-packages -m something - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 26, 2014, at 02:16 PM, Piotr Ożarowski wrote: >then --dont-blame-us has to mean removing /usr/local from sys.path Right. -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140326103152.08f25...@anarchist.wooz.org
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 26, 2014, at 12:15 AM, Scott Kitterman wrote: >If I've install a package and it's upgraded (this is for the system, not for >any kind of virtualized/isolated environment), I would find it quite >surprising and unfortunate that it upgraded itself from an external source. IMO, if you've apt-get installed a package, this goes in /usr/lib and no way would pip --upgrade upgrade that. IOW, for modules installed with apt-get, apt-get is the only way to upgrade it. It follows then, that if you've sudo pip installed something into /usr/local, then pip --upgrade *can* upgrade that. The tricky place is whether you will allow an apt-get installed module to be sudo pip --upgraded by installing the newer externally sourced package into /usr/local. That would require /usr/local to come first on sys.path, which is the case currently at least on Debian. I do think this would require a --dont-blame-us switch analogous to -s/$PYTHONNOUSERSITE. We'd want to recommend adding that switch to shebang lines for system services and scripts, much like we already do with -Es. For complete isolation, -I should imply this new switch too. In fact, given that we *already* include /usr/local/lib/pythonX.Y/dist-packages on sys.path, I think it's a deficiency that we don't define such a switch (but we'd want to pick something that would align with an upstream choice for this switch). Cheers, -Barry signature.asc Description: PGP signature
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 26, 2014, at 09:24 AM, Donald Stufft wrote: >In my half formed idea in my head the way it’d work is there’d be a >vendor-packages directory where downstream can install things to, and a flag >to the interpreter to remove the typical site-packages. So then you’d get >something like: > >python -I —no-site-packages -m something There has to be a short option for --no-site-packages (or whatever) so that it will work with shebang lines, where we already recommend -Es. But also, -I should imply this new option for full isolation. Which means for Python 3.4 and beyond we should be recommending system services and scripts add -I to the shebang line instead of -Es. Then we'd get this new /usr/local isolation switch for free. -Barry signature.asc Description: PGP signature
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 26, 2014, at 10:35 AM, Barry Warsaw wrote: > On Mar 26, 2014, at 09:24 AM, Donald Stufft wrote: > >> In my half formed idea in my head the way it’d work is there’d be a >> vendor-packages directory where downstream can install things to, and a flag >> to the interpreter to remove the typical site-packages. So then you’d get >> something like: >> >> python -I —no-site-packages -m something > > There has to be a short option for --no-site-packages (or whatever) so that it > will work with shebang lines, where we already recommend -Es. Yea I just didn’t feel like thinking up a short option :) > But also, -I > should imply this new option for full isolation. Not sure about this, I don’t think I saw the original discussion but it looks like -I is to prevent the user from injecting malicious code (so it removes env vars, the user site packages, the current dir, etc). I don’t think that something installed by pip by the system administrator falls under that. > > Which means for Python 3.4 and beyond we should be recommending system > services and scripts add -I to the shebang line instead of -Es. Then we'd get > this new /usr/local isolation switch for free. > > -Barry - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 26, 2014, at 04:12 AM, Bohuslav Kabrda wrote: >This really starts to look like we should all join one mailing list and >discuss this in one thread :) What about the pypa ML? I guess joining one >more ML is worth this... Yep, I think pypa is probably the right mailing list to take this to. -Barry signature.asc Description: PGP signature
Re: Python 3.4 and ensurepip (rehashed, long)
[Barry Warsaw, 2014-03-26] > I do think this would require a --dont-blame-us switch analogous to > -s/$PYTHONNOUSERSITE. We'd want to recommend adding that switch to shebang > lines for system services and scripts, much like we already do with -Es. For > complete isolation, -I should imply this new switch too. What if system administrator wants to extend some system services/scripts/etc. via installing plugins/additional libraries to /usr/local/pythonX.Y/dist-packages? Isn't that what this dir is for? If we start disabling this path in all our script, what's the point in having it? Instead of adding hacks, we should educate users (IMO pip should at least warn when used with root rights, etc.). Adding another /usr/local/pythonX.Y/user-packages to sys.path (with a switch in our /usr/bin/scripts that disables it) sounds tempting, but I doubt we'll be able to force all these tools out there (that overwrite our files) to use it. -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140326150452.gg30...@sts0.p1otr.com
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 26, 2014, at 10:42 AM, Donald Stufft wrote: >> But also, -I should imply this new option for full isolation. > >Not sure about this, I don’t think I saw the original discussion but it looks >like -I is to prevent the user from injecting malicious code (so it removes >env vars, the user site packages, the current dir, etc). I don’t think that >something installed by pip by the system administrator falls under that. Here's the issue where -I was discussed (it wasn't a PEP): http://bugs.python.org/issue16499 Here, "isolation mode" means specifically isolating the Python interpreter from "bad stuff a user could do" to their environment. So I guess it is questionable whether to lump the sysadmin in that category ;). OT1H, a distro wants to pretty clear about what the expected environment for system services and scripts is. OTOH, we probably need to give the sysadmin the benefit of the doubt and allow them to modify their system for their needs, but then take responsibility for those changes. So I can see the argument that -I should not include --dont-blame-us. With a short option, it wouldn't be difficult to recommend "full isolation" in shebang lines with -I@ (where @ == short option). -Barry signature.asc Description: PGP signature
Re: Python 3.4 and ensurepip (rehashed, long)
On Fri, Mar 21, 2014 at 01:23:53PM +0100, Piotr Ożarowski wrote:
> [Barry Warsaw, 2014-03-19]
> > TL;DR: Let's re-enable the ensurepip module in Python 3.4, and possibly
> >address some usability issues. We should descend en masse on Montreal
> > and
> >stage a revolt at Pycon. :)
>
> IMO our ensurepip.py should look similar to this:
>
> | try:
> | from pip import *
> | except ImportError:
> | raise Exception('Please ask administrator to install python3-pip
> package.'
> | ' Note that installing packages system-wide using pip
> is'
> | ' considered harmful, please do not report Python
> related'
> | ' bugs in Debian bugtracker if you decide to do that.')
>
> IMO we should warn users that they can^W will break their systems and
> `sudo pip install ...` should raise an exception if
> --i-will-not-blame-debian option is not enabled.
I recently had to clean up after a sysadmin, previously explicitly
warned not to, used "sudo pip install ...". I think the option should
require an argument which must match with a randomly generated string.
That way only the NSA could ever actually use the option as only they
own all the random numbers.
--
Brian Sutherland
--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
https://lists.debian.org/20140326161329.GA25017@Brians-MacBook-Air.local
