[Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
Several significant changes in this revision: - scope narrowed to just Python 2.7 plus permission for commercial redistributors to use the same strategy in their long term support releases - far more explicit that this is about inviting potential corporate contributors to address the situation for the benefit of the overall Python ecosystem, not offering to fix it for them for free - clarified that third party integration testing services would need to be updated to support testing against multiple Python 2.7 minor releases - explicit sections on why I don't think the status quo is sustainable, why I don't think Python 2.8 would actually solve the problem, and why I think a PyPI based solution not only wouldn't solve the problem, but would be rather difficult to get working in the first place - be completely explicit that I am *not* speaking on behalf of Red Hat at this point and have no authority to make commitments on their behalf. Instead, I'm looking for upstream consensus that 1) this is a genuine problem that needs to be solved; 2) we're open to corporate assistance in solving it; and 3) we have a pretty good idea what help we actually want. If all that happens, *then* I can take up the issue internally to try to get us some help in maintaining the proposed solution (hopefully other folks with corporate influence can do the same, and we may actually get some ongoing assistance with upstream maintenance out of this, rather than having our downstream redistributors continue to take us for granted). Diff: http://hg.python.org/peps/rev/2e82209dda21 Updated web version: http://www.python.org/dev/peps/pep-0466/ Advance warning: while I was able to get this revision turned around pretty quickly, future revisions are likely to take a fair bit longer. It was already a rather busy month before I decided to start this discussion on top of everything else :) Cheers, Nick. PEP: 466 Title: Network Security Enhancement Exception for Python 2.7 Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan , Status: Draft Type: Informational Content-Type: text/x-rst Created: 23-Mar-2014 Post-History: 23-Mar-2014 Abstract Most CPython tracker issues are classified as errors in behaviour or proposed enhancements. Most patches to fix behavioural errors are applied to all active maintenance branches. Enhancement patches are restricted to the default branch that becomes the next Python version. This cadence works reasonably well during Python's normal 18-24 month feature release cycle, which is still applicable to the Python 3 series. However, the age of the standard library in Python 2 has now reached a point where it is sufficiently far behind the state of the art in network security protocols for it to be causing real problems in commercial use cases where upgrading to Python 3 in the near term may not be practical. Accordingly, this PEP relaxes the normal restrictions by allowing enhancements to be applied in Python 2.7 maintenance releases for standard library components that have implications for the overall security of the internet. In particular, the exception will apply to: * the ``ssl`` module * the ``hashlib`` module * the ``hmac`` module * the ``sha`` module (Python 2 only) * the components of other networking modules that make use of these modules * the components of the ``random`` and ``os`` modules that are relevant to cryptographic applications * the version of OpenSSL bundled with the binary installers Proposed backports for these modules will still need to undergo normal backwards compatibility assessments, but new features will be permitted where appropriate, making it easier to implement secure networked software in Python, even for software that needs to remain compatible with older feature releases of Python. While this PEP does not make any changes to the core development team's handling of security-fix-only branches that are no longer in active maintenance, it *does* recommend that commercial redistributors providing extended support periods for the Python standard library either adopt a similar approach to ensuring that the secure networking infrastructure keeps pace with the evolution of the internet, or else disclaim support for the use of older versions in roles that involving connecting directly to the public internet. Exemption Policy Under this policy, the following network security related modules are granted a blanket exemption to the restriction against adding new features in maintenance releases, for the purpose of keeping their APIs aligned with their counterparts in the latest feature release of Python 3: * the ``ssl`` module * the ``hashlib`` module * the ``hmac`` module * the ``sha`` module (Python 2 only) This exemption applies to *all* proposals to backport backwards compatible changes in these modules to Python 2.7 maintenance releases. This choice is made deliberately to ensure that the "feature or fix?" argument isn't simply repla
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
On Mar 23, 2014, at 3:07 AM, Nick Coghlan wrote: > Several significant changes in this revision: > > - scope narrowed to just Python 2.7 plus permission for commercial > redistributors to use the same strategy in their long term support > releases > - far more explicit that this is about inviting potential corporate > contributors to address the situation for the benefit of the overall > Python ecosystem, not offering to fix it for them for free > - clarified that third party integration testing services would need > to be updated to support testing against multiple Python 2.7 minor > releases For what it’s worth, I have an outstanding PR against Travis CI that would make this trivial for them at least. They are a pretty popular CI service especially for OSS projects. I made that PR for unrelated reasons but it could at least serve as a template for other projects to do the same thing. > - explicit sections on why I don't think the status quo is > sustainable, why I don't think Python 2.8 would actually solve the > problem, and why I think a PyPI based solution not only wouldn't solve > the problem, but would be rather difficult to get working in the first > place > - be completely explicit that I am *not* speaking on behalf of Red Hat > at this point and have no authority to make commitments on their > behalf. Instead, I'm looking for upstream consensus that 1) this is a > genuine problem that needs to be solved; 2) we're open to corporate > assistance in solving it; and 3) we have a pretty good idea what help > we actually want. If all that happens, *then* I can take up the issue > internally to try to get us some help in maintaining the proposed > solution (hopefully other folks with corporate influence can do the > same, and we may actually get some ongoing assistance with upstream > maintenance out of this, rather than having our downstream > redistributors continue to take us for granted). > > Diff: http://hg.python.org/peps/rev/2e82209dda21 > Updated web version: http://www.python.org/dev/peps/pep-0466/ > > Advance warning: while I was able to get this revision turned around > pretty quickly, future revisions are likely to take a fair bit longer. > It was already a rather busy month before I decided to start this > discussion on top of everything else :) > > Cheers, > Nick. > > > PEP: 466 > Title: Network Security Enhancement Exception for Python 2.7 > Version: $Revision$ > Last-Modified: $Date$ > Author: Nick Coghlan , > Status: Draft > Type: Informational > Content-Type: text/x-rst > Created: 23-Mar-2014 > Post-History: 23-Mar-2014 > > > Abstract > > > Most CPython tracker issues are classified as errors in behaviour or > proposed enhancements. Most patches to fix behavioural errors are > applied to all active maintenance branches. Enhancement patches are > restricted to the default branch that becomes the next Python version. > > This cadence works reasonably well during Python's normal 18-24 month > feature release cycle, which is still applicable to the Python 3 series. > However, the age of the standard library in Python 2 has now reached a point > where it is sufficiently far behind the state of the art in network security > protocols for it to be causing real problems in commercial use cases > where upgrading to Python 3 in the near term may not be practical. > > Accordingly, this PEP relaxes the normal restrictions by allowing > enhancements to be applied in Python 2.7 maintenance releases for standard > library components that have implications for the overall security of the > internet. In particular, the exception will apply to: > > * the ``ssl`` module > * the ``hashlib`` module > * the ``hmac`` module > * the ``sha`` module (Python 2 only) > * the components of other networking modules that make use of these modules > * the components of the ``random`` and ``os`` modules that are relevant to > cryptographic applications > * the version of OpenSSL bundled with the binary installers > > Proposed backports for these modules will still need to undergo normal > backwards compatibility assessments, but new features will be permitted where > appropriate, making it easier to implement secure networked software in > Python, even for software that needs to remain compatible with older feature > releases of Python. > > While this PEP does not make any changes to the core development team's > handling of security-fix-only branches that are no longer in active > maintenance, it *does* recommend that commercial redistributors providing > extended support periods for the Python standard library either adopt a > similar approach to ensuring that the secure networking infrastructure > keeps pace with the evolution of the internet, or else disclaim support > for the use of older versions in roles that involving connecting > directly to the public internet. > > > Exemption Policy > > > Under this policy, the following network security related modules are
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
On Sun, Mar 23, 2014 at 6:07 PM, Nick Coghlan wrote: > And that's just three of the highest profile open source projects that > make heavy use of Python. Given the likely existence of large amounts of > legacy code that lacks the kind of automated regression test suite needed > to help support a migration from Python 2 to Python 3. The key point of > this PEP is that those situations affect more people than just the > developers and users of the affected application: their existence becomes > something that developers of secure networked services need to take into > account as part of their security design. Grammatical point: The sentence beginning "Given..." doesn't seem complete. ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 453 (Explicit bootstrapping of pip in Python installations) - slight typo
Hi. Not really sure where to report this - missing closing parentheses in the PEP text at the end of the second paragraph in section 'Implementation strategy' http://legacy.python.org/dev/peps/pep-0453/#id35 > and would not try to contact PyPI (instead installing directly > from the private wheel files. Hope this helps. Best regards, Jurko Gospodnetić ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
Am 23.03.14 08:07, schrieb Nick Coghlan: > Several significant changes in this revision: > > - scope narrowed to just Python 2.7 plus permission for commercial > redistributors to use the same strategy in their long term support > releases Thanks; the rationale is now much clearer, and also indicates yet another alternative path: fork Python. The PEP indicates that vendors are likely to fork Python anyway (as they always did, in a small scale). This could become more official and coordinated. Create an "enhanced TLS" clone of cpython, and start applying changes to 2.6, 2.7, and any other branches you consider relevant. Keep it merged with the cpython code base. This model has worked for many years for Stackless Python. Then, vendors have the choice of either releasing from the official CPython repository, or from the enhanced TLS one. All reasoning on application-level feature testing still applies: applications would have to feature-test whether they run on python.org release, or on an enhanced release. For Windows in particular, it would be up to ActiveState to decide whether they build binaries from python.org, or from the enhanced TLS repo. They could actually opt to provide both, leaving the choice to users. Even if this notion of forking is not acceptable, I suggest that you could still start working on the code in a separate clone, and the decision on the PEP could be deferred until a proposed implementation is ready. I see it as a risk of the PEP that the implementation might not be available before May 2015, in which case the PEP would become irrelevant. Regards, Martin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23.03.2014 02:33, Brett Cannon wrote: > Now I have been reading this thread on my phone and I only have cursory > understanding of what failure ssl has had as of late, so this might be > stupid, but what if in Python 3.5 we made it so people passed in an > explicit SSL object into the relevant APIs and threw an exception if one > wasn't provided when trying to access something requiring SSL? Then the > ssl module would start existing on PyPI to make sure it can evolve > faster than every 18 months? I want to deprecate all SSL-related arguments except for the context argument. Several modules still except key and cert parameters. Do you want to make the context parameter a required parameter? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Hi, 2014-03-22 22:11 GMT+01:00 Nick Coghlan : > In particular, the exception will apply to: > > * the ``ssl`` module > * the ``hashlib`` module > * the ``hmac`` module > * the ``sha`` module (Python 2 only) > * the components of other networking modules that make use of these modules > * the components of the ``random`` and ``os`` modules that are relevant to > cryptographic applications > * the version of OpenSSL bundled with the binary installers The current Python development cycle is too slow to track latest security recommandations. Python 2.7 is now very far from Python 3.4, so backportng so many features is very complex and error-prone. IMO A third-party module backporting all these security enhancements would be easy to develop. There are already M2Crypto, cryptography and PyOpenSSL modules for example. You may reuse them. The drawback is that applications would be benefit immediatly from this work, they should be modified to use the new module. But usually, developers who care of security are able to do these modifications. The PEP is very generic. Can you try to list all missing security features of old Python versions? Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Quoting Victor Stinner : The drawback is that applications would be benefit immediatly from this work, they should be modified to use the new module. But usually, developers who care of security are able to do these modifications. I think asking developers to make significant modifications to their code is besides the point of the PEP. However, if they are willing to make changes, I'd still recommend that they port their code to Python 3, as that is the better long-term investment. Regards, Martin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
Hi, 2014-03-23 11:17 GMT+01:00 : > Quoting Victor Stinner : >> The drawback is that applications would be benefit immediatly from >> this work, they should be modified to use the new module. But usually, >> developers who care of security are able to do these modifications. > > I think asking developers to make significant modifications to their > code is besides the point of the PEP. I expect something like replacing "import ssl" with "from safestdlib import ssl", same for other modules. FYI I backported some Python 3.3 modules for Python 2.7 for my Trollius project, and it was not so hard. For ssl.SSLContext, it's an empty class, it's just a wrapper to ssl.wrap_socket() which raises errors when unsupported features are used. http://trollius.readthedocs.org/#backports I wrote Trollius to port OpenStack to Python 3, not to defer the port :-) > However, if they are willing > to make changes, I'd still recommend that they port their code to > Python 3, as that is the better long-term investment. It's not always possible. For example, the OpenStack projet still have +30 dependencies which are not Python 3 compatible yet. https://wiki.openstack.org/wiki/Python3#Dependencies We are porting these dependencies, but it may still take one year to port the whole OpenStack project to Python 3. Come to help us :) Sorry, it's maybe not fair to take the worst example (OpenStack) :-) Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 Mar 2014 20:33, "Victor Stinner" wrote: > > Sorry, it's maybe not fair to take the worst example (OpenStack) :-) I suspect the Fedora/RHEL/CentOS ecosystem is going to make OpenStack look like a relatively simple port, and backwards compatibility constraints mean that already defined RHEL/CentOS releases simply won't be forward ported at all. I confess I hadn't fully connected the relevant dots until after writing the PEP this weekend, so I think I'm going to be having some interesting conversations with colleagues about that over the next couple of weeks and at PyCon :) Cheers, Nick. > > Victor > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
On 23 Mar 2014 18:42, Martin v. Löwis wrote: > > Am 23.03.14 08:07, schrieb Nick Coghlan: > > Several significant changes in this revision: > > > > - scope narrowed to just Python 2.7 plus permission for commercial > > redistributors to use the same strategy in their long term support > > releases > > Thanks; the rationale is now much clearer, and also indicates yet > another alternative path: fork Python. > > The PEP indicates that vendors are likely to fork Python anyway > (as they always did, in a small scale). This could become more official > and coordinated. Create an "enhanced TLS" clone of cpython, and start > applying changes to 2.6, 2.7, and any other branches you consider > relevant. Keep it merged with the cpython code base. This model > has worked for many years for Stackless Python. > > Then, vendors have the choice of either releasing from the official > CPython repository, or from the enhanced TLS one. All reasoning on > application-level feature testing still applies: applications would > have to feature-test whether they run on python.org release, or > on an enhanced release. > > For Windows in particular, it would be up to ActiveState to decide > whether they build binaries from python.org, or from the enhanced > TLS repo. They could actually opt to provide both, leaving the choice > to users. > > Even if this notion of forking is not acceptable, I suggest > that you could still start working on the code in a separate clone, > and the decision on the PEP could be deferred until a proposed > implementation is ready. I see it as a risk of the PEP that the > implementation might not be available before May 2015, in which > case the PEP would become irrelevant. Yes, a 2.7-enhanced-tls clone is definitely a good notion. Your suggestion to recast the entire PEP along those lines, so we always retain the option of choosing between them, also sounds plausible. Cheers, Nick. > > Regards, > Martin > > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sat, Mar 22, 2014 at 11:31 PM, Terry Reedy wrote: > The download page for the final 2.7.z maintenance release could say > something like "We recommend that you move to the most recent Python 3 > version if at all possible. If you cannot do that and you want to use Python > to run a server on the public internet, we urge you to instead use the > latest version of ServerPython 2.7.1s. This series is based on Python 2.7.z > but has been and will continue to be enhanced with security features > backported from Python 3." I'm unclear how this would be better than just biting the bullet and making a 2.8 release. On the one hand, the 2.7.x number suggests (based on the existing release protocol) that it should be a drop-in replacement for earlier 2.7 micro releases. On the other hand, calling it something like "ServerPython" implies that it's not necessary for network client applications, when, if I read the PEP correctly, it most certainly would be. If you create a 2.8 release which is restricted to just the topic areas of the PEP (that is, no other stuff backported from 3.x, no requirement to add other non-security bug fixes, etc), the incremented minor version number tells people that a bit of extra care is required to upgrade. The lack of change in the code base outside the security apparatus should make update pretty trivial for most every non-networked application. If the PEP or something like it is approved, the work is still going to have to be done, no matter what you call it. Why not be transparent about it? Skip ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
On Sun, 23 Mar 2014 17:07:24 +1000 Nick Coghlan wrote: > Another more critical example is the lack of SSL hostname matching in the > Python 2 standard library - it is currently necessary to rely on a third > party library, such as ``requests`` or ``backports.ssl_match_hostname`` to > obtain that functionality in Python 2. Do note that match_hostname() is a pure Python function and is easy to paste into your own code (if you don't want to pull in a dependency). It doesn't need SSLContext or any other recent stuff, just a certificate dict which Python 2.x is already able to provide (SSLSocket.getpeercert()). > Firstly, this PEP encompasses a non-trivial portion of the standard library. > It's not just the underlying SSL support, but also the libraries for other > network protocols like HTTP, FTP, IMAP, and POP3 that integrate with the > SSL infrastructure to provide secure links, and that's just the protocols > in the standard library. It's still not obvious what you are proposing to do with these other libraries. If you are proposing to validate certs against system CAs and check hostnames by default - you are going to break compatibility for a lot of current uses. As Martin I think it would be easier to reason about a concrete backport proposal. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 22 March 2014 21:11, Nick Coghlan wrote:
> Full PEP included inline below, and available in more readable form at
> http://www.python.org/dev/peps/pep-0466/
Disclaimer: I pretty much don't use 2.x, or write server-type
software, so my practical requirement for the changes proposed here is
negligible.
Having thought about the points in the discussion so far, it seems to me:
1. The only real issue is 2.7, so the PEP should limit itself to "how
do we deal with 2.7".
2. The PEP talks vaguely about commercial support for the security
changes. I'm not quite sure what's being said here - I suspect Nick
has some concrete input from Red Hat here, but he's trying to keep
from being too specific in the PEP. I don't think that's necessarily
helpful. If the proposal is "Companies who want backported security
features can provide patches and this is how python-dev will evaluate
them" then let's say that. At the moment, there's a feel in the PEP
that python-dev are proposing to do the backports, and I'm not sure
that's true. As written, the PEP risks exposing python-dev to
accusations of not having delivered the improved security that the PEP
promises.
3. If companies want specific security enhancements, there's nothing
to stop them developing a patch, writing a PEP and proposing it to
python-dev. The first such patch would trigger pretty much the current
debate (is it OK to add an enhancement to 2.7, because it is for
security reasons?) but with the benefit of being about something
specific. Further proposals along the same lines could reference the
first PEP as justification for allowing subsequent security
enhancements.
4. A process like that described in (3) could happen right now. Why
isn't it? Presumably the need for such enhancements is there (or
there'd be no need for this PEP) so is it simply that companies like
Red Hat don't know that they have this option? Would it be better
simply to publish something explaining "How commercial vendors can
work with python-dev to deliver long-term support of Python 2.7".?
5. If the driver for this is Linux vendor support for 2.7, who is
going to provide the Windows/OSX/Solaris/etc implementations of new
features? Is "nobody" (implying that new features could end up being
Linux-only) an acceptable answer? Will the burden of adding
cross-platform support be on python-dev? Will patches be rejected
unless they cover all supported platforms (and will this be an
unacceptable burden for companies like Red Hat to take on)?
The situation with Python 2.7 has been understood for many years now.
Individual end users may not have thought through the implications and
we have a responsibility to support them. But companies like Red
Hat[1] are fully aware of their own release cycles, and the position
with Python 2.x is public knowledge. They have to take some share of
the blame for not planning for this situation (and they have paying
customers who expect them to handle this, unlike python-dev).
One problem here is that it's not at all clear what counts as "dealing
with a commercial vendor" means in the context of enhancements
(security or otherwise) to Python. We already have a route that's open
to anyone, so what else is needed? And why?
Most of the above is directed at the "commercial vendors can help out"
aspect of the PEP. With regard to the wider suggestion that security
enhancements be allowed for 2.7, we've been debating similar questions
in various forms for *ages*. There's been talk of a 2.8 release. There
have been ongoing discussions about helping people move to Python 3.
There have been many discussions about whether it's OK for
applications to remain on Python 2.7 (all of which said "yes, it's
fine" - maybe the security fix aspect makes that statement a little
less clear-cut). The fact that we haven't felt the need thus far to
change policy on 2.7 says that ("commercial vendors need it" aspects
aside) this is just another cycle of that same debate.
Paul
[1] I seem to be picking on Red Hat here. I'm not - it's just that the
RHEL support cycle is the most-often quoted example of commercial
support for Python 2.x, so I'm using that as my example.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
On 23 March 2014 07:07, Nick Coghlan wrote: > Advance warning: while I was able to get this revision turned around > pretty quickly, future revisions are likely to take a fair bit longer. > It was already a rather busy month before I decided to start this > discussion on top of everything else :) Ha - you produced this update while I was still thinking about the first draft, and it arrived (unnoticed) while I was writing my response. Sorry if some or all of my points in the other email are now irrelevant as a result. I'll try to read and assess this version before responding - so feel free to take longer over the next revision:-) Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 23 March 2014 at 04:32:17, Terry Reedy ([email protected](mailto:[email protected])) wrote: > Instead, I think the PEP should propose a special series of server > enhancement releases that are based on the final 2.7 maintenance release > (2.7.8 or 2.7.9) but which have have a different application-specific > enhancement policy. This is an interesting idea. My biggest problem with it is that, at least with the ssl library, these aren’t server-only problems. If we suggest that they are, we end up in the same position we’re in right now (that is, hurting the internet). For example, Python 2.7’s ssl module lacks the OP_NO_COMPRESSION option for OpenSSL, meaning that the application is at the mercy of the server to determine whether it’s vulnerable to the CRIME attack. Given that all modern browsers already disable TLS compression, we can assume that lots of server admins haven’t bothered disabling it on their end. This leaves pretty much anyone using HTTPS, client or server, on Python 2.7 at risk of the CRIME attack. This isn’t a server-only problem, so I feel like limiting the fixes to a ‘server’ release is not good enough. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Sun, 23 Mar 2014 07:29:07 + Cory Benfield wrote: > On 23 March 2014 at 04:32:17, Terry Reedy > ([email protected](mailto:[email protected])) wrote: > > Instead, I think the PEP should propose a special series of server > > enhancement releases that are based on the final 2.7 maintenance release > > (2.7.8 or 2.7.9) but which have have a different application-specific > > enhancement policy. > > This is an interesting idea. My biggest problem with it is that, at least > with the ssl library, these aren’t server-only problems. If we suggest that > they are, we end up in the same position we’re in right now (that is, hurting > the internet). > > For example, Python 2.7’s ssl module lacks the OP_NO_COMPRESSION option for > OpenSSL, This is easy to change in a bugfix release, though. Someone just has to open an issue and write a patch. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 23, 2014, at 11:34 AM, Antoine Pitrou wrote: > On Sun, 23 Mar 2014 07:29:07 + > Cory Benfield wrote: >> On 23 March 2014 at 04:32:17, Terry Reedy >> ([email protected](mailto:[email protected])) wrote: >>> Instead, I think the PEP should propose a special series of server >>> enhancement releases that are based on the final 2.7 maintenance release >>> (2.7.8 or 2.7.9) but which have have a different application-specific >>> enhancement policy. >> >> This is an interesting idea. My biggest problem with it is that, at least >> with the ssl library, these aren’t server-only problems. If we suggest that >> they are, we end up in the same position we’re in right now (that is, hurting >> the internet). >> >> For example, Python 2.7’s ssl module lacks the OP_NO_COMPRESSION option for >> OpenSSL, > > This is easy to change in a bugfix release, though. Someone just has to > open an issue and write a patch. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io I already did open an issue and write a patch :) There’s someone on that issue saying that flipping that without a way to flip it back would break their application. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 3/23/2014 11:37 AM, Donald Stufft wrote: > > On Mar 23, 2014, at 11:34 AM, Antoine Pitrou wrote: > >> On Sun, 23 Mar 2014 07:29:07 + >> Cory Benfield wrote: >>> This is an interesting idea. My biggest problem with it is that, at least >>> with the ssl library, these aren’t server-only problems. If we suggest that >>> they are, we end up in the same position we’re in right now (that is, >>> hurting >>> the internet). >>> >>> For example, Python 2.7’s ssl module lacks the OP_NO_COMPRESSION option for >>> OpenSSL, >> >> This is easy to change in a bugfix release, though. Someone just has to >> open an issue and write a patch. > > I already did open an issue and write a patch :) > > There’s someone on that issue saying that flipping that without a way to flip > it back > would break their application. http://bugs.python.org/issue20994, if anyone is looking for it. Eric. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] OP_NO_COMPRESSION
On Sun, 23 Mar 2014 11:37:25 -0400 Donald Stufft wrote: > > I already did open an issue and write a patch :) > > There’s someone on that issue saying that flipping that without a way to flip > it back > would break their application. You're right, I had forgotten about that :-) I'd be surprised if disabling compression could altogether break an application. It may make it less efficient, though, if it relied on TLS to compress data for it (which doesn't sound like a good thing to rely on, anyway). Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OP_NO_COMPRESSION
On Mar 23, 2014, at 11:46 AM, Antoine Pitrou wrote: > On Sun, 23 Mar 2014 11:37:25 -0400 > Donald Stufft wrote: >> >> I already did open an issue and write a patch :) >> >> There’s someone on that issue saying that flipping that without a way to >> flip it back >> would break their application. > > You're right, I had forgotten about that :-) > > I'd be surprised if disabling compression could altogether break an > application. It may make it less efficient, though, if it relied on TLS > to compress data for it (which doesn't sound like a good thing to rely > on, anyway). > > Regards > > Antoine. > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io I tend to agree about that, but I was mostly echoing the fact that they’ve expressed that sentiment. Without SSLContext I couldn’t think of a reasonable way to enable them to turn it on/off that didn’t involve a whole new API just for some versions of 2.7. This seemed to me like a wholly worse idea than back porting SSLContext :/ - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OP_NO_COMPRESSION
On 23/03/2014 15:46, Antoine Pitrou wrote: On Sun, 23 Mar 2014 11:37:25 -0400 Donald Stufft wrote: I already did open an issue and write a patch :) There’s someone on that issue saying that flipping that without a way to flip it back would break their application. You're right, I had forgotten about that :-) I'd be surprised if disabling compression could altogether break an application. It may make it less efficient, though, if it relied on TLS to compress data for it (which doesn't sound like a good thing to rely on, anyway). Regards Antoine. I've looked at the issue and can't find any such comment. The closest is from http://bugs.python.org/issue20994#msg214246 "Also some users will absolutely want to manually re-enable compression, please don't disable it entirely." Chinese whispers? :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7
On Mar 23, 2014, at 9:13 AM, Antoine Pitrou wrote: > On Sun, 23 Mar 2014 17:07:24 +1000 > Nick Coghlan wrote: >> Another more critical example is the lack of SSL hostname matching in the >> Python 2 standard library - it is currently necessary to rely on a third >> party library, such as ``requests`` or ``backports.ssl_match_hostname`` to >> obtain that functionality in Python 2. > > Do note that match_hostname() is a pure Python function and is easy to > paste into your own code (if you don't want to pull in a dependency). > It doesn't need SSLContext or any other recent stuff, just a > certificate dict which Python 2.x is already able to provide > (SSLSocket.getpeercert()). So the problem with match_hostname is that it’s a security sensitive function, there have already been at least one fix to it because of it doing something incorrectly. Advocating users to copy it into their own code case typically means that it’ll get copied once and forgotten. So for any security updates in the future they are unlikely to get those. It seems like the danger of _adding_ things like that is pretty minimal. > >> Firstly, this PEP encompasses a non-trivial portion of the standard library. >> It's not just the underlying SSL support, but also the libraries for other >> network protocols like HTTP, FTP, IMAP, and POP3 that integrate with the >> SSL infrastructure to provide secure links, and that's just the protocols >> in the standard library. > > It's still not obvious what you are proposing to do with these other > libraries. If you are proposing to validate certs against system CAs and > check hostnames by default - you are going to break compatibility for a > lot of current uses. > > As Martin I think it would be easier to reason about a concrete backport > proposal. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OP_NO_COMPRESSION
On Mar 23, 2014, at 11:55 AM, Mark Lawrence wrote: > On 23/03/2014 15:46, Antoine Pitrou wrote: >> On Sun, 23 Mar 2014 11:37:25 -0400 >> Donald Stufft wrote: >>> >>> I already did open an issue and write a patch :) >>> >>> There’s someone on that issue saying that flipping that without a way to >>> flip it back >>> would break their application. >> >> You're right, I had forgotten about that :-) >> >> I'd be surprised if disabling compression could altogether break an >> application. It may make it less efficient, though, if it relied on TLS >> to compress data for it (which doesn't sound like a good thing to rely >> on, anyway). >> >> Regards >> >> Antoine. > > I've looked at the issue and can't find any such comment. The closest is > from http://bugs.python.org/issue20994#msg214246 "Also some users will > absolutely want to manually re-enable compression, please don't disable it > entirely." Chinese whispers? :) > > -- > My fellow Pythonistas, ask not what our language can do for you, ask what you > can do for our language. > > Mark Lawrence > > --- > This email is free from viruses and malware because avast! Antivirus > protection is active. > http://www.avast.com > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io Ah, maybe he told me that in IRC then, I actually know him and was arguing with him about it there. Sorry I got my streams crossed. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] On porting to Python 3 as the answer
This really upset me: On Sun, Mar 23, 2014 at 3:17 AM, wrote: > I think asking developers to make significant modifications to their > code is besides the point of the PEP. However, if they are willing > to make changes, I'd still recommend that they port their code to > Python 3, as that is the better long-term investment. > This is a completely unrealistic form of wishful thinking, and repeating it won't make it more true. At Dropbox I work with a large group of very capable developers on several large code bases that are currently in 2.7. We are constantly changing our code to make it more secure (there are several teams specifically in charge of that). And yet porting to Python 3 is completely out of scope, for a variety of reasons. Please stop your wishful thinking. (TBH, I expect that none of the changes to Python 2.7 under consideration would make any difference for the security of Dropbox. But neither would switching to Python 3.) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On porting to Python 3 as the answer
Am 23.03.14 17:22, schrieb Guido van Rossum: > At Dropbox I work with a large group of very capable developers on > several large code bases that are currently in 2.7. We are constantly > changing our code to make it more secure (there are several teams > specifically in charge of that). And yet porting to Python 3 is > completely out of scope, for a variety of reasons. > > Please stop your wishful thinking. I can stop expressing it; I don't think I can stop wishing it :-) If it's really unrealistic that Dropbox will ever port the code to Python 3, would you then think that Python 3 is a doomed project, since it won't ever see significant usage? Regards, Martin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On porting to Python 3 as the answer
On Mar 23, 2014, at 12:33 PM, Martin v. Löwis wrote: > Am 23.03.14 17:22, schrieb Guido van Rossum: >> At Dropbox I work with a large group of very capable developers on >> several large code bases that are currently in 2.7. We are constantly >> changing our code to make it more secure (there are several teams >> specifically in charge of that). And yet porting to Python 3 is >> completely out of scope, for a variety of reasons. >> >> Please stop your wishful thinking. > > I can stop expressing it; I don't think I can stop wishing it :-) > > If it's really unrealistic that Dropbox will ever port the code > to Python 3, would you then think that Python 3 is a doomed project, > since it won't ever see significant usage? I think expecting every production instance of Python 2.7 to port is unrealistic. New projects will start to get written in Python 3, some existing projects will get ported over time. But I doubt there is ever going to be some mass exodus where everyone suddenly starts moving to Python 3. For people/companies/projects where Python 2 is currently working fine and porting would be a significant effort they have to decide between adding more value to their project through bug fixes, new features, etc or essentially sinking cost into Porting to Python 3 which their end users will not benefit from in a very large way. It’s hard to make that business case for existing code, especially given that for a lot of the truly hard parts of porting to Python 3 there’s no way to port module by module so you have to basically do an entire code base at once. This adds extra churn, increases the risk for new bugs, and is quite an annoying process. That doesn’t mean Python 3 is bad or is inherently a doomed project, but it’s essentially a new, but very similar, language. > > Regards, > Martin > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On porting to Python 3 as the answer
On Sun, Mar 23, 2014 at 9:33 AM, "Martin v. Löwis" wrote: > Am 23.03.14 17:22, schrieb Guido van Rossum: > > At Dropbox I work with a large group of very capable developers on > > several large code bases that are currently in 2.7. We are constantly > > changing our code to make it more secure (there are several teams > > specifically in charge of that). And yet porting to Python 3 is > > completely out of scope, for a variety of reasons. > > > > Please stop your wishful thinking. > > I can stop expressing it; I don't think I can stop wishing it :-) > > If it's really unrealistic that Dropbox will ever port the code > to Python 3, would you then think that Python 3 is a doomed project, > since it won't ever see significant usage? > No, it's just that the timescale is drastically different. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: #20145: assert[Raises|Warns]Regex now raise TypeError on bad regex.
On Sun, 23 Mar 2014 20:47:28 +0100 (CET) r.david.murray wrote: > http://hg.python.org/cpython/rev/ec556e45641a > changeset: 89936:ec556e45641a > user:R David Murray > date:Sun Mar 23 15:08:43 2014 -0400 > summary: > #20145: assert[Raises|Warns]Regex now raise TypeError on bad regex. > > Previously a non-string, non-regex second argument could cause the test > to always pass. It seems like this would be useful to fix in 3.4 too. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On porting to Python 3 as the answer
On 24 Mar 2014 03:48, "Guido van Rossum" wrote: > > On Sun, Mar 23, 2014 at 9:33 AM, "Martin v. Löwis" wrote: >> >> Am 23.03.14 17:22, schrieb Guido van Rossum: >> > At Dropbox I work with a large group of very capable developers on >> > several large code bases that are currently in 2.7. We are constantly >> > changing our code to make it more secure (there are several teams >> > specifically in charge of that). And yet porting to Python 3 is >> > completely out of scope, for a variety of reasons. >> > >> > Please stop your wishful thinking. >> >> I can stop expressing it; I don't think I can stop wishing it :-) >> >> If it's really unrealistic that Dropbox will ever port the code >> to Python 3, would you then think that Python 3 is a doomed project, >> since it won't ever see significant usage? > > > No, it's just that the timescale is drastically different. Yep, and Paul was ultimately right in guessing that part of my motivation here was seeing trouble on the horizon in terms of RH's ability to affordably sustain our own long term support commitments for the Python 2 series. That said, I didn't actually realise the full implications for us until *after* writing, publishing and receiving feedback on the PEP, though - otherwise I would likely have done things in a different order. Isn't open source fun? :) So I'm now going to switch my focus on this topic to the Fedora community for a while: I think the second draft of the PEP is in a pretty reasonable state, and from my perspective, anything we come up with is going to have to at least pass muster with the Fedora Engineering Steering Committee for it to actually solve the upcoming supportability problem on that side of things. It would be good if folks engaged with other downstream redistributors could also get them to take a look and provide feedback. A plan we like, but key redistributors choose not to go along with, would be rather missing the point... Cheers, Nick. > > -- > --Guido van Rossum (python.org/~guido) > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 23, 2014, at 01:01 AM, Antoine Pitrou wrote: >But enforcing "secure by default" can by construction break backwards >compatibility, which is the very reason we are so conservative with >such changes. Also, many developers who are stuck on Python 2 have already evaluated, designed, and implemented workarounds for security issues in ancient stdlib code. You have to be very careful that any changes in some future 2.7 stdlib secure-by-default release doesn't break those workarounds. That's a "trick question" too because you can't know all of them. I didn't read the PEP until just now, so I never saw the first draft. As written it still makes me uncomfortable because as Antoine says, lots of changes could be classified as "security related" and we definitely don't want this PEP to be used as a wedge to make a wink-wink-nudge-nudge release of Python 2.8. I think the key point for consumers of Python has to be *predictability*. -Barry ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 3/23/2014 3:29 AM, Cory Benfield wrote: On 23 March 2014 at 04:32:17, Terry Reedy ([email protected](mailto:[email protected])) wrote: Instead, I think the PEP should propose a special series of server enhancement releases that are based on the final 2.7 maintenance release (2.7.8 or 2.7.9) but which have have a different application-specific enhancement policy. This is an interesting idea. My biggest problem with it is that, at least with the ssl library, these aren’t server-only problems. If we suggest that they are, we end up in the same position we’re in right now (that is, hurting the internet). If I understood the problem and pep too narrowly, substitute 'security' or whatever phrase you want for 'server'. My last sentence was an admission that some details of what I said would need to be changed. The key idea is to admit the what is being proposed is a minor fork of 2.7 (somewhat like Stackless 2.7) that might break code and which would require separate code testing -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: #20145: assert[Raises|Warns]Regex now raise TypeError on bad regex.
On Sun, 23 Mar 2014 21:43:14 +0100, Antoine Pitrou wrote: > On Sun, 23 Mar 2014 20:47:28 +0100 (CET) > r.david.murray wrote: > > http://hg.python.org/cpython/rev/ec556e45641a > > changeset: 89936:ec556e45641a > > user:R David Murray > > date:Sun Mar 23 15:08:43 2014 -0400 > > summary: > > #20145: assert[Raises|Warns]Regex now raise TypeError on bad regex. > > > > Previously a non-string, non-regex second argument could cause the test > > to always pass. > > It seems like this would be useful to fix in 3.4 too. You will note that the change got a porting note in What's New. It will only break tests (that are currently no-ops), not code, so I'd be fine with backporting it, but the original issue did not call for that. If we fix it in 3.4, should we fix it in 2.7 as well? --David ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: #20145: assert[Raises|Warns]Regex now raise TypeError on bad regex.
On Sun, 23 Mar 2014 19:44:42 -0400 "R. David Murray" wrote: > On Sun, 23 Mar 2014 21:43:14 +0100, Antoine Pitrou > wrote: > > On Sun, 23 Mar 2014 20:47:28 +0100 (CET) > > r.david.murray wrote: > > > http://hg.python.org/cpython/rev/ec556e45641a > > > changeset: 89936:ec556e45641a > > > user:R David Murray > > > date:Sun Mar 23 15:08:43 2014 -0400 > > > summary: > > > #20145: assert[Raises|Warns]Regex now raise TypeError on bad regex. > > > > > > Previously a non-string, non-regex second argument could cause the test > > > to always pass. > > > > It seems like this would be useful to fix in 3.4 too. > > You will note that the change got a porting note in What's New. It will > only break tests (that are currently no-ops), not code, so I'd be fine > with backporting it, but the original issue did not call for that. Exposing buggy tests sounds like a good thing to me :-) > If we fix it in 3.4, should we fix it in 2.7 as well? If the *Regex methods are there, yes, IMO. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 24 Mar 2014 09:27, "Barry Warsaw" wrote: > > On Mar 23, 2014, at 01:01 AM, Antoine Pitrou wrote: > > >But enforcing "secure by default" can by construction break backwards > >compatibility, which is the very reason we are so conservative with > >such changes. > > Also, many developers who are stuck on Python 2 have already evaluated, > designed, and implemented workarounds for security issues in ancient stdlib > code. You have to be very careful that any changes in some future 2.7 stdlib > secure-by-default release doesn't break those workarounds. That's a "trick > question" too because you can't know all of them. > > I didn't read the PEP until just now, so I never saw the first draft. As > written it still makes me uncomfortable because as Antoine says, lots of > changes could be classified as "security related" and we definitely don't want > this PEP to be used as a wedge to make a wink-wink-nudge-nudge release of > Python 2.8. Hence why the proposal mostly *isn't* scoped by "security related". It is scoped by "affects the security design of other systems, including systems not written in Python". The core problem to be solved is ensuring that the "cold, dead hand" of the Python 2 standard library doesn't hold back the evolution of internet security standards. I'm starting to think it makes sense to drop the higher level abstractions from the proposal, though. Once the core SSL support issue is addressed, it's comparatively straightforward to handle any other cases as PyPI modules if needed. pip, for example, already bundles requests to handle HTTPS, but that still relies on the stdlib SSL support. > I think the key point for consumers of Python has to be *predictability*. Agreed. That's a key part of why the proposal is mainly about syncing certain key modules with their Python 3 counterparts, rather than piecemeal backports. That way, all you need to know is "the SSL, hashlib and hmac modules are kept in sync with Python 3 feature releases, but use the same default settings as the original Python 2.7 release". A partial backport is actually *harder* for users to deal with (another reason I'm quite amenable to dropping that part of the proposal). Cheers, Nick. > > -Barry > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 3/23/2014 9:00 AM, Skip Montanaro wrote: On Sat, Mar 22, 2014 at 11:31 PM, Terry Reedy wrote: The download page for the final 2.7.z maintenance release could say something like "We recommend that you move to the most recent Python 3 version if at all possible. If you cannot do that and you want to use Python to run a server on the public internet, we urge you to instead use the latest version of ServerPython 2.7.1s. This series is based on Python 2.7.z but has been and will continue to be enhanced with security features backported from Python 3." I'm unclear how this would be better than just biting the bullet and making a 2.8 release. The first step is to 'bite the bullet' and admit that the proposal is for a fork of 2.7. (Martin use that word today and pointed out the analogy with Stackless Python.) I decided not to suggest '2.8' because a) we said there would be no 2.8 (though that could be reversed); b) once we say '2.8', there would be a mountain of proposals to change this and that; c) people will expect the change from 2.7 to 2.8 to be something like previous deltas, but it will not be; d) the security fork might have a change that would normally require a deprecation period, but there might not be; and e) once a 2.8 were released, it too would be closed to enhancements, so that 2.9, 2.10, 2.11.. would eventually be needed. I mentioned this as one possible solution, but one I do not like. In summary, "2.8 is to 2.7 as 2.7 is to 2.6, etcetera" would not be true, so lets not pretend or mislead people that is would be. The fork series, not being a regular CPython series, would not be subject to the regular CPython rule of needing a new minor version number for each set of enhancements and spacing the sets 18 to 24 months apart. > On the one hand, the 2.7.x number suggests (based on the existing release protocol) that it should be a drop-in replacement for earlier 2.7 micro releases. No CPython x.y.z has a two-digit number for z. I know that this is a subtle hint, which is why I also propose a new name. I do not think that anyone expects Stackless Python 2.7 to be an exact drop-in replacement for any 2.7.z, although it is for code that is not affected by its specialized alterations. > On the other hand, calling it something like "ServerPython" implies that it's not necessary for network client applications, when, if I read the PEP correctly, it most certainly would be. Consider 'ServerPython' a placeholder for a better name. In my penultimate sentence "There are still details to be filled in or altered," I was specifically thinking of this. If you create a 2.8 release which is restricted to just the topic areas of the PEP (that is, no other stuff backported from 3.x, no requirement to add other non-security bug fixes, etc), the incremented minor version number tells people that a bit of extra care is required to upgrade. The lack of change in the code base outside the security apparatus should make update pretty trivial for most every non-networked application. If the PEP or something like it is approved, the work is still going to have to be done, no matter what you call it. Why not be transparent about it? We agree on being transparent, even if we have different ideas on exactly what needs to be made clear. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 23, 2014, at 08:00 AM, Skip Montanaro wrote: >I'm unclear how this would be better than just biting the bullet and >making a 2.8 release. On the one hand, the 2.7.x number suggests >(based on the existing release protocol) that it should be a drop-in >replacement for earlier 2.7 micro releases. On the other hand, calling >it something like "ServerPython" implies that it's not necessary for >network client applications, when, if I read the PEP correctly, it >most certainly would be. It seems to me that the problem the PEP addresses can largely be confined to the Python 2.7 standard library and the fact that it's bundled with the language. I want to stick to our no-Python-2.8 pledge, but I wonder if there isn't a middle ground where we fork the stdlib into a separate branch, and apply security specific changes there. Python 2.7.x will always be the "standard stdlib". We would never release a specific Python 2.7 + "security stdlib" release, but downstream developers would be able to overlay this forked stdlib on top of the standard one. Volunteers or corporate sponsors could distribute binary installers with this combination of pure Python 2.7 language + "security enhanced stdlib", and Linux distros could do the necessary building and distributing for their own platforms if they so desired. The trick is what do you call this new combination, how do you invoke it, and how do you keep it distinct and independent of the system's standard Python 2.7? Maybe this is some scent of Python 2.8 by another name, but let's never call it Python 2.8. -Barry ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
> On Mar 23, 2014, at 7:03 PM, Barry Warsaw wrote: > >> On Mar 23, 2014, at 08:00 AM, Skip Montanaro wrote: >> >> I'm unclear how this would be better than just biting the bullet and >> making a 2.8 release. On the one hand, the 2.7.x number suggests >> (based on the existing release protocol) that it should be a drop-in >> replacement for earlier 2.7 micro releases. On the other hand, calling >> it something like "ServerPython" implies that it's not necessary for >> network client applications, when, if I read the PEP correctly, it >> most certainly would be. > > It seems to me that the problem the PEP addresses can largely be confined to > the Python 2.7 standard library and the fact that it's bundled with the > language. I want to stick to our no-Python-2.8 pledge, but I wonder if there > isn't a middle ground where we fork the stdlib into a separate branch, and > apply security specific changes there. > > Python 2.7.x will always be the "standard stdlib". We would never release a > specific Python 2.7 + "security stdlib" release, but downstream developers > would be able to overlay this forked stdlib on top of the standard one. > Volunteers or corporate sponsors could distribute binary installers with this > combination of pure Python 2.7 language + "security enhanced stdlib", and > Linux distros could do the necessary building and distributing for their own > platforms if they so desired. > > The trick is what do you call this new combination, how do you invoke it, and > how do you keep it distinct and independent of the system's standard Python > 2.7? > > Maybe this is some scent of Python 2.8 by another name, but let's never call > it Python 2.8. > It seems like this and the fork idea are both jumping through hoops to avoid the inevitable - a 2.7 security release or a 2.8 security only release. Especially as I know nick isn't the only one looking to hire FTEs for this specific work as it's actively hurting service providers/distributions and others. If we call it a security only release, and scope/review all patches to that it seems that it's pretty clear what the release is for. 2.x is going to be in the wild for at least another decade as these large legacy codebases are functioning, profitable and dedicating teams of engineers to port them to 3.x doesn't make sense to the business. As new services and things come online we'll see the gradual movement to 3 accelerate especially as people realize clients and service talk better on an interface such as http rather than large monolithic tragedies. > -Barry > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mon, Mar 24, 2014 at 11:03 AM, Barry Warsaw wrote: > Python 2.7.x will always be the "standard stdlib". We would never release a > specific Python 2.7 + "security stdlib" release, but downstream developers > would be able to overlay this forked stdlib on top of the standard one. > Volunteers or corporate sponsors could distribute binary installers with this > combination of pure Python 2.7 language + "security enhanced stdlib", and > Linux distros could do the necessary building and distributing for their own > platforms if they so desired. > > The trick is what do you call this new combination, how do you invoke it, and > how do you keep it distinct and independent of the system's standard Python > 2.7? Easy. Just set PYTHONPATH to import the SEPython [1] lib ahead of the standard lib. Then you can go back to the standard 2.7 (if you want to) by unsetting PYTHONPATH. It'd be nice if SEPython defined a modified sys.version for clarity, but otherwise, it'd be a vanilla Python 2.7. ChrisA [1] By analogy with SELinux ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 3/23/2014 8:03 PM, Barry Warsaw wrote: On Mar 23, 2014, at 08:00 AM, Skip Montanaro wrote: I'm unclear how this would be better than just biting the bullet and making a 2.8 release. On the one hand, the 2.7.x number suggests (based on the existing release protocol) that it should be a drop-in replacement for earlier 2.7 micro releases. On the other hand, calling it something like "ServerPython" implies that it's not necessary for network client applications, when, if I read the PEP correctly, it most certainly would be. It seems to me that the problem the PEP addresses can largely be confined to the Python 2.7 standard library and the fact that it's bundled with the language. I want to stick to our no-Python-2.8 pledge, but I wonder if there isn't a middle ground where we fork the stdlib into a separate branch, and apply security specific changes there. Python 2.7.x will always be the "standard stdlib". We would never release a specific Python 2.7 + "security stdlib" release, but downstream developers would be able to overlay this forked stdlib on top of the standard one. Volunteers or corporate sponsors could distribute binary installers with this combination of pure Python 2.7 language + "security enhanced stdlib", and Linux distros could do the necessary building and distributing for their own platforms if they so desired. The trick is what do you call this new combination, I have already agree that 'ServerPython' is just a placeholder. > how do you invoke it, and how do you keep it distinct and > independent of the system's standard Python 2.7? Maybe this is some scent of Python 2.8 by another name, but let's never call it Python 2.8. I agree, no only because of the 'pledge', but because it is the wrong model. Nick's latest paraphrase of where he thinks his proposal is heading is "the SSL, hashlib and hmac modules are kept in sync with Python 3 feature releases, but use the same default settings as the original Python 2.7 release". '2.8' would imply a broader focus than just 3 modules, and it would only work for the first of a series of enhancement releases. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On 3/23/2014 7:48 PM, Nick Coghlan wrote: Agreed. That's a key part of why the proposal is mainly about syncing certain key modules with their Python 3 counterparts, rather than piecemeal backports. That way, all you need to know is "the SSL, hashlib and hmac modules are kept in sync with Python 3 feature releases, but use the same default settings as the original Python 2.7 release". If you restrict the proposal to just those three modules, then the proposal could be to just add three new modules to 2.7: ssl3, hashlib3, and hmac3. Each would be regularly updated backports of the 3.x modules with two differences: defaults for settings that exist in 2.7 would remain as in 2.7 and would not be updated*, and each module would get a .py3_ver x.y.z attribute so code could be conditioned on which backport version it is running with. * though users would be encouraged to use the most secure settings. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 23, 2014, at 8:31 PM, Jesse Noller wrote: > > >> On Mar 23, 2014, at 7:03 PM, Barry Warsaw wrote: >> >>> On Mar 23, 2014, at 08:00 AM, Skip Montanaro wrote: >>> >>> I'm unclear how this would be better than just biting the bullet and >>> making a 2.8 release. On the one hand, the 2.7.x number suggests >>> (based on the existing release protocol) that it should be a drop-in >>> replacement for earlier 2.7 micro releases. On the other hand, calling >>> it something like "ServerPython" implies that it's not necessary for >>> network client applications, when, if I read the PEP correctly, it >>> most certainly would be. >> >> It seems to me that the problem the PEP addresses can largely be confined to >> the Python 2.7 standard library and the fact that it's bundled with the >> language. I want to stick to our no-Python-2.8 pledge, but I wonder if there >> isn't a middle ground where we fork the stdlib into a separate branch, and >> apply security specific changes there. >> >> Python 2.7.x will always be the "standard stdlib". We would never release a >> specific Python 2.7 + "security stdlib" release, but downstream developers >> would be able to overlay this forked stdlib on top of the standard one. >> Volunteers or corporate sponsors could distribute binary installers with this >> combination of pure Python 2.7 language + "security enhanced stdlib", and >> Linux distros could do the necessary building and distributing for their own >> platforms if they so desired. >> >> The trick is what do you call this new combination, how do you invoke it, and >> how do you keep it distinct and independent of the system's standard Python >> 2.7? >> >> Maybe this is some scent of Python 2.8 by another name, but let's never call >> it Python 2.8. >> > > It seems like this and the fork idea are both jumping through hoops to avoid > the inevitable - a 2.7 security release or a 2.8 security only release. > Especially as I know nick isn't the only one looking to hire FTEs for this > specific work as it's actively hurting service providers/distributions and > others. > > If we call it a security only release, and scope/review all patches to that > it seems that it's pretty clear what the release is for. > > 2.x is going to be in the wild for at least another decade as these large > legacy codebases are functioning, profitable and dedicating teams of > engineers to port them to 3.x doesn't make sense to the business. As new > services and things come online we'll see the gradual movement to 3 > accelerate especially as people realize clients and service talk better on an > interface such as http rather than large monolithic tragedies. > > >> -Barry >> ___ >> Python-Dev mailing list >> [email protected] >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io I agree, the bulk of the alternative suggestions feel more like trying to adhere to a policy for policy’s sake rather than actually figure out what is best for the users. Adding new APIs to 2.7 feels to me like a pretty backwards compatible thing to be honest. The biggest contenders are things like SSLContext and match_hostname. The PEP still says things must be backwards compatible so we can't change existing APIs in a way that the backported things aren't opt in. I mean how likely is it that there is code out there relying on the fact that SSLContext doesn't exist other than as a detection to determine if they can use SSLContext? - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: #20145: assert[Raises|Warns]Regex now raise TypeError on bad regex.
On 3/23/2014 7:47 PM, Antoine Pitrou wrote: On Sun, 23 Mar 2014 19:44:42 -0400 "R. David Murray" wrote: On Sun, 23 Mar 2014 21:43:14 +0100, Antoine Pitrou wrote: On Sun, 23 Mar 2014 20:47:28 +0100 (CET) r.david.murray wrote: ... Previously a non-string, non-regex second argument could cause the test to always pass. It seems like this would be useful to fix in 3.4 too. You will note that the change got a porting note in What's New. It will only break tests (that are currently no-ops), not code, so I'd be fine with backporting it, but the original issue did not call for that. Exposing buggy tests sounds like a good thing to me :-) If we fix it in 3.4, should we fix it in 2.7 as well? If the *Regex methods are there, yes, IMO. I agree. Since I and others am writing tests for all current versions of Idle, I would like a buggy test to fail no matter where it is first tested. Currently, I would tend to start with 3.4. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 24, 2014, at 11:38 AM, Chris Angelico wrote: >Easy. Just set PYTHONPATH to import the SEPython [1] lib ahead of the >standard lib. Then you can go back to the standard 2.7 (if you want >to) by unsetting PYTHONPATH. > >It'd be nice if SEPython defined a modified sys.version for clarity, >but otherwise, it'd be a vanilla Python 2.7. That's certainly more in the direction of what I think is an appropriate upstream solution. The thing is, there isn't one single "what's best for users" resolution. There are many, many competing requirements and I think it will be difficult to satisfy everyone. I'm particularly sensitive to complaints of unexpected changes between micro releases. -Barry ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mar 23, 2014, at 9:31 PM, Barry Warsaw wrote: > On Mar 24, 2014, at 11:38 AM, Chris Angelico wrote: > >> Easy. Just set PYTHONPATH to import the SEPython [1] lib ahead of the >> standard lib. Then you can go back to the standard 2.7 (if you want >> to) by unsetting PYTHONPATH. >> >> It'd be nice if SEPython defined a modified sys.version for clarity, >> but otherwise, it'd be a vanilla Python 2.7. > > That's certainly more in the direction of what I think is an appropriate > upstream solution. > > The thing is, there isn't one single "what's best for users" resolution. > There are many, many competing requirements and I think it will be difficult > to satisfy everyone. I'm particularly sensitive to complaints of unexpected > changes between micro releases. If it’s too much for a micro release call it 2.8 or 2.9 or something and give it a super limited scope. Proposals like modifying the sys.path to opt into the backported things or having an official unofficial stdlib just adds more things downstream users have to actually test against and more possible configurations or Pythons. Right now users have a singular method for determining what the runtime environment looks like for Python, the version. There are processes around selecting different Python versions for things, upgrading etc. This isn’t a new thing for users. Suddenly having to mess with PYTHONPATH or having to figure out if their distro used the “official stdlib” or the “official unofficial stdlib” just adds massive confusion. Things like ssl3 etc are “ok” but less than optimal IMO because one of the benefits of adding SSLContext and such to Python 2.x is that you can write code that works in 2.x and 3.x without having to special case 2.7.7 or 2.8 or whatever. However if you just add ssl3 then now you have a third path through the code that you have to deal with. > > -Barry > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/donald%40stufft.io - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements
On Mon, Mar 24, 2014 at 1:34 PM, Donald Stufft wrote: > Right now users have a singular method for determining what the runtime > environment looks like for Python, the version. There are processes around > selecting different Python versions for things, upgrading etc. This isn’t > a new thing for users. Suddenly having to mess with PYTHONPATH or > having to figure out if their distro used the “official stdlib” or the > “official > unofficial stdlib” just adds massive confusion. Various distros of Linux ship modified versions of things. On my current system (Debian Wheezy), the apt-gettable Python is tagged "2.7.3-4+deb7u1", so there are already at least some patches that aren't in vanilla 2.7.3. Presumably the distro could happily just ship the enhanced stdlib, and you don't need to mess with PYTHONPATH; and a modified sys.version or other detection method could be used to probe if you need to be sure. As has already been pointed out, this can already happen, but in an ad-hoc way. Making it official or semi-official would mean that a script written for Debian's "Python 2.7.10" would run on Red Hat's "Python 2.7.10", which would surely be an advantage. ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
