Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Le Thu, 21 Feb 2013 00:30:56 +0100, Christian Heimes a écrit : > Am 21.02.2013 00:08, schrieb Antoine Pitrou: > > Not everyone is a security nuts. > > But, but, but ... it's fun to be paranoid! You get so many new > potential enemies. :) If you like being paranoid, there are other things than security to be paranoid about: reference cycles, performance on micro-benchmarks, memory consumption of docstrings, etc. :-) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Am 21.02.2013 10:23, schrieb Antoine Pitrou: > If you like being paranoid, there are other things than security to > be paranoid about: reference cycles, performance on micro-benchmarks, > memory consumption of docstrings, etc. :-) snappy(__doc__)? http://code.google.com/p/snappy/ Christian ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Am 21.02.2013 08:42, schrieb Antoine Pitrou: > Sure, but in many instances, rebooting a machine is not > business-threatening. You will have a couple of minutes' downtime and > that's all. Which is why the attack must be repeated many times to be a > major annoyance. Is this business-threatening enough? https://pypi.python.org/pypi/defusedxml#external-entity-expansion-remote * An attacker can circumvent firewalls and gain access to restricted resources as all the requests are made from an internal and trustworthy IP address, not from the outside. * An attacker can abuse a service to attack, spy on or DoS your servers but also third party services. The attack is disguised with the IP address of the server and the attacker is able to utilize the high bandwidth of a big machine. * An attacker can exhaust additional resources on the machine, e.g. with requests to a service that doesn't respond or responds with very large files. * An attacker may gain knowledge, when, how often and from which IP address a XML document is accessed. * An attacker could send mail from inside your network if the URL handler supports smtp:// URIs. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Le Thu, 21 Feb 2013 11:18:35 +0100, Christian Heimes a écrit : > Am 21.02.2013 08:42, schrieb Antoine Pitrou: > > Sure, but in many instances, rebooting a machine is not > > business-threatening. You will have a couple of minutes' downtime > > and that's all. Which is why the attack must be repeated many times > > to be a major annoyance. > > Is this business-threatening enough? > > https://pypi.python.org/pypi/defusedxml#external-entity-expansion-remote You haven't proved that these were actual threats, nor how they actually worked. I'm gonna remain skeptical if there isn't anything more precise than "It highly depends on the parser and the application what kind of exploit is possible". Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
On Feb 21, 2013, at 5:32 AM, Antoine Pitrou wrote: > Le Thu, 21 Feb 2013 11:18:35 +0100, > Christian Heimes a écrit : >> Am 21.02.2013 08:42, schrieb Antoine Pitrou: >>> Sure, but in many instances, rebooting a machine is not >>> business-threatening. You will have a couple of minutes' downtime >>> and that's all. Which is why the attack must be repeated many times >>> to be a major annoyance. >> >> Is this business-threatening enough? >> >> https://pypi.python.org/pypi/defusedxml#external-entity-expansion-remote > > You haven't proved that these were actual threats, nor how they > actually worked. I'm gonna remain skeptical if there isn't anything > more precise than "It highly depends on the parser and the application > what kind of exploit is possible". > > Regards > > Antoine. > I guess someone need to write a proof of concept exploit for you and release it into the wild. Ok > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Le Thu, 21 Feb 2013 06:05:52 -0500, Jesse Noller a écrit : > On Feb 21, 2013, at 5:32 AM, Antoine Pitrou > wrote: > > > Le Thu, 21 Feb 2013 11:18:35 +0100, > > Christian Heimes a écrit : > >> Am 21.02.2013 08:42, schrieb Antoine Pitrou: > >>> Sure, but in many instances, rebooting a machine is not > >>> business-threatening. You will have a couple of minutes' downtime > >>> and that's all. Which is why the attack must be repeated many > >>> times to be a major annoyance. > >> > >> Is this business-threatening enough? > >> > >> https://pypi.python.org/pypi/defusedxml#external-entity-expansion-remote > > > > You haven't proved that these were actual threats, nor how they > > actually worked. I'm gonna remain skeptical if there isn't anything > > more precise than "It highly depends on the parser and the > > application what kind of exploit is possible". > > > > Regards > > > > Antoine. > > > > I guess someone need to write a proof of concept exploit for you and > release it into the wild. I don't know whether you are trying to be ironic but, for the record, proof of concepts needn't be "released into the wild" as long as they exist. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Am 21.02.2013 11:32, schrieb Antoine Pitrou: > You haven't proved that these were actual threats, nor how they > actually worked. I'm gonna remain skeptical if there isn't anything > more precise than "It highly depends on the parser and the application > what kind of exploit is possible". https://bitbucket.org/tiran/defusedxml/src/82f4037464418bf11ea734969b7ca1c193e6ed91/other/python-external.py?at=default $ ./python-external.py REQUEST: Aachen RESPONSE: - The weather in Aachen is terrible. ]> &passwd; RESPONSE: - Unknown city root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/bin/sh bin:x:2:2:bin:/bin:/bin/sh sys:x:3:3:sys:/dev:/bin/sh sync:x:4:65534:sync:/bin:/bin/sync games:x:5:60:games:/usr/games:/bin/sh man:x:6:12:man:/var/cache/man:/bin/sh lp:x:7:7:lp:/var/spool/lpd:/bin/sh mail:x:8:8:mail:/var/mail:/bin/sh news:x:9:9:news:/var/spool/news:/bin/sh uucp:x:10:10:uucp:/var/spool/uucp:/bin/sh proxy:x:13:13:proxy:/bin:/bin/sh www-data:x:33:33:www-data:/var/www:/bin/sh backup:x:34:34:backup:/var/backups:/bi REQUEST: http://hg.python.org/cpython/raw-file/a11ddd687a0b/Lib/test/dh512.pem";> ]> &url; RESPONSE: - Unknown city -BEGIN DH PARAMETERS- MEYCQQD1Kv884bEpQBgRjXyEpwpy1obEAxnIByl6ypUM2Zafq9AKUJsCRtMIPWak XUGfnHy9iUsiGSa6q6Jew1XpKgVfAgEC -END DH PARAMETERS- These are the 512 bit DH parameters from "Assigned Number for SKIP Protocols" (http://www.skip-vpn.org/spec/numbers.html). See there for how they were generated. Note that g is not a generator, but this is not a problem since p is a safe prime. Q.E.D. Christian ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Am 21.02.2013 12:16, schrieb Antoine Pitrou: > I don't know whether you are trying to be ironic but, for the record, > proof of concepts needn't be "released into the wild" as long as they > exist. Fun fact: In fact the abbreviation 'ap' doesn't stand for 'Antoine Pitrou' but for 'antipole'. I'm a bit paranoid and overcautious. Antoine acts as my antipole and counter balance. Together we make a fairly good team. :) Christian ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Le Thu, 21 Feb 2013 13:19:54 +0100, Christian Heimes a écrit : > Am 21.02.2013 12:16, schrieb Antoine Pitrou: > > I don't know whether you are trying to be ironic but, for the > > record, proof of concepts needn't be "released into the wild" as > > long as they exist. > > Fun fact: > > In fact the abbreviation 'ap' doesn't stand for 'Antoine Pitrou' but > for 'antipole'. I'm a bit paranoid and overcautious. Antoine acts as > my antipole and counter balance. Together we make a fairly good > team. :) Not really, since I'm overcautious towards paranoid people. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Le Thu, 21 Feb 2013 13:04:59 +0100, Christian Heimes a écrit : > Am 21.02.2013 11:32, schrieb Antoine Pitrou: > > You haven't proved that these were actual threats, nor how they > > actually worked. I'm gonna remain skeptical if there isn't anything > > more precise than "It highly depends on the parser and the > > application what kind of exploit is possible". > > https://bitbucket.org/tiran/defusedxml/src/82f4037464418bf11ea734969b7ca1c193e6ed91/other/python-external.py?at=default > > $ ./python-external.py [snip] Again, this requires that your attacker can directly feed XML to the system *and* read the response. Not every computer is a public Internet server. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
On Thu, Feb 21, 2013 at 6:35 AM, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 02/20/2013 09:08 PM, Barry Warsaw wrote: >> On Feb 21, 2013, at 10:38 AM, Nick Coghlan wrote: >> >>> - make it possible to enable safer behaviour globally in at least >>> 2.7 and 3.3 (and perhaps in 2.6 and 3.2 security releases as well) >> >> I want to be fairly conservative with 2.6.9. > > I believe that the same rationale should apply as that for adding hash > randomization in 2.6.8: this is at least as bad a vulnerability, with > many more vectors of attack. FYI the hash randomization is broken (it only allows 256 really different hashes) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
On Thu, Feb 21, 2013 at 9:29 AM, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 02/21/2013 01:53 AM, Antoine Pitrou wrote: >> On Thu, 21 Feb 2013 11:37:47 +1100 Steven D'Aprano >> wrote: >>> >>> It's easy to forget that malware existed long before the Internet. >>> The internet is just a transmission vector, it is not the source of >>> malicious files. The source of malicious files is *other people*, >>> and unless you never use XML files you didn't generate yourself, you >>> cannot completely trust the source. You might trust your colleagues >>> to not *intentionally* pass you a malicious XML file, but they may >>> still do so accidentally. >> >> That's in theory very nice, but in practice security in everyday >> computing hasn't really been a concern before the massification of >> Internet access. >> >> (yes, there have been viruses on mainstream platforms such as the >> Amiga, but it was pretty minor compared to nowadays, and nobody cared >> about potential DoS attacks for example) >> >> So, as for XML files, we are talking about a DoS vulnerability. It >> will take more than a single file to make a DoS attack really >> annoying, which means the attacker must pollute the source of those >> XML files in a systemic way. It's not "a single XML file will smuggle >> confidential data out of the building". > > Antoine, > > A single, small,, malicious XML file can kill a machine (not just the > process parsing it) by sucking all available RAM. We are talking hard > lockup, reboot-to-fix-it sorts of DOC here. Er no. We're talking about running out of RAM. Any reasonable person would already have a limit one way or another (rlimits anyone). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Jesse Noller writes:
> I guess someone need to write a proof of concept exploit for you
> and release it into the wild.
This is a bit ridiculous. This stuff looks easy enough that surely
Christian's post informed any malicious body who didn't already know
how to do it. If the exploit matters, it's already in the wild.
("Hey, didja know that an XML processor that expands entities does so
recursively?" "Uh-oh ")
Yeah, there's a problem here. But ... as far as I can see all the
exploits suggested (including those Christian provided in
python-external.py) require either blindly processing text from
requests received off the Internet as XML, or an attacker capable of
doing something equivalent to replacing a Python library.
I certainly think defusedxml is a valuable contribution, and not just
for security nuts. But to quote from Christian's own README (warning:
taken out of context to make *my* point):
7. These are features but they may introduce exploitable holes, see
`Other things to consider`_
I'd like to see a little (well, to be honest, a *lot*) more analysis
of the kind Fred Drake implicitly suggests:
Doing so *will* be backward incompatible, and I'm not sure there's
a good way to gauge the extent of the breakage.
before making these restrictions the default. Eg, 40 entity
indirections in a single expansion (defusedxml's default maximum) may
seem like a lot, but I've seen some pretty complex expressions built
as entities that recurse three or four levels. Of course, that was a
while ago, and today most of the entities would be replaced by actual
characters. Nevertheless, I bet those legacy expressions break the 40
indirection limit, or, rather, the limit would break them.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
On Thu, Feb 21, 2013 at 9:23 AM, Stephen J. Turnbull wrote:
> Jesse Noller writes:
>
> > I guess someone need to write a proof of concept exploit for you
> > and release it into the wild.
>
> This is a bit ridiculous. This stuff looks easy enough that surely
> Christian's post informed any malicious body who didn't already know
> how to do it. If the exploit matters, it's already in the wild.
> ("Hey, didja know that an XML processor that expands entities does so
> recursively?" "Uh-oh ")
>
Just to clarify for my own curiosity. These attacks (e.g.
http://en.wikipedia.org/wiki/Billion_laughs) have been known and public
since 2003?
Eli
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
Am 21.02.2013 19:39, schrieb Eli Bendersky: > Just to clarify for my own curiosity. These attacks (e.g. > http://en.wikipedia.org/wiki/Billion_laughs) have been known and public > since 2003? Correct, see https://pypi.python.org/pypi/defusedxml#synopsis third paragraph. All XML attacks in my analysis are well known for years, billion laughs for about a decade. As far as I know it's the first time somebody has compiled and published a detailed list of vulnerabilities in Python's XML libraries. However I'm not the only one. OpenStack and Django were contacted by several people in the past few weeks, too. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python
On Thu, Feb 21, 2013 at 11:12 AM, Christian Heimes wrote: > Am 21.02.2013 19:39, schrieb Eli Bendersky: > > Just to clarify for my own curiosity. These attacks (e.g. > > http://en.wikipedia.org/wiki/Billion_laughs) have been known and public > > since 2003? > > Correct, see https://pypi.python.org/pypi/defusedxml#synopsis third > paragraph. All XML attacks in my analysis are well known for years, > billion laughs for about a decade. > > As far as I know it's the first time somebody has compiled and published > a detailed list of vulnerabilities in Python's XML libraries. However > I'm not the only one. OpenStack and Django were contacted by several > people in the past few weeks, too. > Thanks, Christian. I think this should put the urgency of the fix into context. While I agree that we should work on making future versions resilient by default, I have doubts about the urgency of back-patching existing, in-mainteinance-mode stable versions with something that's not opt-in. Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] xml.sax and xml.dom fetch DTDs by default (was XML DoS vulnerabilities and exploits in Python)
Perhaps related to the discussion of denial-of-service vulnerabilities is the matter of controlling access to remote resources. I suppose that after the following bug was closed, no improvements were made to the standard library: http://bugs.python.org/issue2124 Do Python programs still visit the W3C site millions of times every day to download DTDs that they are not, by default, able to remember from their last visit? Paul ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] xml.sax and xml.dom fetch DTDs by default
Am 22.02.2013 00:47, schrieb Paul Boddie: > Perhaps related to the discussion of denial-of-service vulnerabilities is the > matter of controlling access to remote resources. I suppose that after the > following bug was closed, no improvements were made to the standard library: > > http://bugs.python.org/issue2124 > > Do Python programs still visit the W3C site millions of times every day to > download DTDs that they are not, by default, able to remember from their last > visit? Affirmative for Python 2.6 to 3.4 dev! It's all in my documentation, too. https://pypi.python.org/pypi/defusedxml#python-xml-libraries Christian ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Can't upload to PyPI
Since the PyPI security notice of 2013-02-15 I've been unable to upload to PyPI via "setup.py upload". I changed my password during the grace period, and have reset it, but it's still rejected: Upload failed (401): Incorrect password I can login to PyPI with the password. Can anyone suggest what could be wrong? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't upload to PyPI
This is probably better suited to Catalog-sig but you have to edit your credentials in $HOME/.pypirc On Thu, Feb 21, 2013 at 9:02 PM, MRAB wrote: > Since the PyPI security notice of 2013-02-15 I've been unable to upload > to PyPI via "setup.py upload". > > I changed my password during the grace period, and have reset it, but > it's still rejected: > > Upload failed (401): Incorrect password > > I can login to PyPI with the password. > > Can anyone suggest what could be wrong? > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't upload to PyPI
On 2013-02-22 02:09, Ian Cordasco wrote: On Thu, Feb 21, 2013 at 9:02 PM, MRAB wrote: Since the PyPI security notice of 2013-02-15 I've been unable to upload to PyPI via "setup.py upload". I changed my password during the grace period, and have reset it, but it's still rejected: Upload failed (401): Incorrect password I can login to PyPI with the password. Can anyone suggest what could be wrong? > This is probably better suited to Catalog-sig but you have to edit > your credentials in $HOME/.pypirc > Are any other changes needed in .pypirc, _apart_ from the password? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't upload to PyPI
On Thu, Feb 21, 2013 at 9:27 PM, MRAB wrote: > On 2013-02-22 02:09, Ian Cordasco wrote: >> >> On Thu, Feb 21, 2013 at 9:02 PM, MRAB wrote: >>> >>> Since the PyPI security notice of 2013-02-15 I've been unable to upload >>> to PyPI via "setup.py upload". >>> >>> I changed my password during the grace period, and have reset it, but >>> it's still rejected: >>> >>> Upload failed (401): Incorrect password >>> >>> I can login to PyPI with the password. >>> >>> Can anyone suggest what could be wrong? >>> >> This is probably better suited to Catalog-sig but you have to edit >> your credentials in $HOME/.pypirc >> > Are any other changes needed in .pypirc, _apart_ from the password? > I don't recall needing any other changes. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't upload to PyPI
On 2013-02-22 02:37, Ian Cordasco wrote: On Thu, Feb 21, 2013 at 9:27 PM, MRAB wrote: On 2013-02-22 02:09, Ian Cordasco wrote: On Thu, Feb 21, 2013 at 9:02 PM, MRAB wrote: Since the PyPI security notice of 2013-02-15 I've been unable to upload to PyPI via "setup.py upload". I changed my password during the grace period, and have reset it, but it's still rejected: Upload failed (401): Incorrect password I can login to PyPI with the password. Can anyone suggest what could be wrong? This is probably better suited to Catalog-sig but you have to edit your credentials in $HOME/.pypirc Are any other changes needed in .pypirc, _apart_ from the password? I don't recall needing any other changes. OK, I'll take it to Catalog-sig. Thanks anyway. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
