Re: [PHP-DEV] HashDoS

2016-09-24 0:21 GMT+02:00 Stanislav Malyshev : > Hi! > > > I think there is a confusion about the "servers written in PHP". Those > > applications serves more requests in a single (main) PHP request using > > the even loop. Good examples of that are Aerys or ReactPHP. So we don't > > want to kill

Re: [PHP-DEV] HashDoS

Hi! > I think there is a confusion about the "servers written in PHP". Those > applications serves more requests in a single (main) PHP request using > the even loop. Good examples of that are Aerys or ReactPHP. So we don't > want to kill that main request if one of the handled requests is > malic

Re: [PHP-DEV] HashDoS

On 9/22/16 3:46 AM, Rowan Collins wrote: I think I'm right in saying that the power of the attack comes in the fact that the total time doesn't scale linearly but exponentially. quadratic is what i read in the previous thread, iirc. even so, it's still a useful gain. That doesn't exactly

Re: [PHP-DEV] HashDoS

Hi On Fri, Sep 23, 2016 at 8:47 PM, Stanislav Malyshev wrote: > Hi! > > > That's exactly what we don't want - let the attacker to end our request. > > Why not? What else you can do with this request that has clearly bad and > maliciously constructed data? > > I think there is a confusion about t

Re: [PHP-DEV] HashDoS

Hi! > That's exactly what we don't want - let the attacker to end our request. Why not? What else you can do with this request that has clearly bad and maliciously constructed data? > All other things like string overflows and memory limits are under our > control (e.g. we can set limit on the s

Re: [PHP-DEV] HashDoS

Hi, On Fri, Sep 23, 2016 at 8:16 PM, Stanislav Malyshev wrote: > Hi! > > > We could patch zend_hash.c in two ways: SipHash (slw) or only fatals > > (very bad for e.g. servers written in PHP. When they have to decode some > > Why very bad? > > > JSON, it's trivial for an attacker to crash the

Re: [PHP-DEV] HashDoS

Hi! > We could patch zend_hash.c in two ways: SipHash (slw) or only fatals > (very bad for e.g. servers written in PHP. When they have to decode some Why very bad? > JSON, it's trivial for an attacker to crash them very easily). As that's Fatal error is not crash. It's a normal ending of th

Re: [PHP-DEV] HashDoS

> Am 23.09.2016 um 03:16 schrieb Stanislav Malyshev : > > Hi! > >> The patch is not only targeting JSON. He just used JSON as an example. >> >> Every function generating arrays with keys based on user-defined input >> needs to be updated. > > That looks like a very good way to make a lot of mis

Re: [PHP-DEV] HashDoS

Hi! > The patch is not only targeting JSON. He just used JSON as an example. > > Every function generating arrays with keys based on user-defined input > needs to be updated. That looks like a very good way to make a lot of mistakes, miss a lot of cases and end up playing whack-a-mole with cover

Re: [PHP-DEV] HashDoS

> Am 22.9.2016 um 22:08 schrieb Stanislav Malyshev : > > Hi! > >> Yeah it introduces new functions for updating hash which is used by json >> for updating array and it's also in std object handler which is used when >> updating json object. For some other bits like updating array, it will stay >

Re: [PHP-DEV] HashDoS

Hi! > Yeah it introduces new functions for updating hash which is used by json > for updating array and it's also in std object handler which is used when > updating json object. For some other bits like updating array, it will stay > with fatal. The thing is that json parser can then easily check

Re: [PHP-DEV] HashDoS

Is the XML api also affected by hashdos? cheers, glenn On Thursday, 22 September 2016, Jakub Zelenka wrote: > On Thu, Sep 22, 2016 at 8:13 PM, Niklas Keller > wrote: > > > 2016-09-22 20:10 GMT+02:00 Jakub Zelenka >: > > > >> On Thu, Sep 22, 2016 at 10:54 AM, Rowan Collins < > rowan.coll...@gma

Re: [PHP-DEV] HashDoS

On Thu, Sep 22, 2016 at 8:13 PM, Niklas Keller wrote: > 2016-09-22 20:10 GMT+02:00 Jakub Zelenka : > >> On Thu, Sep 22, 2016 at 10:54 AM, Rowan Collins >> wrote: >> >> > On 22/09/2016 10:48, Jakub Zelenka wrote: >> > >> >> >> >> Nope the point of the Bob's patch is to use graceful handling with

Re: [PHP-DEV] HashDoS

2016-09-22 20:10 GMT+02:00 Jakub Zelenka : > On Thu, Sep 22, 2016 at 10:54 AM, Rowan Collins > wrote: > > > On 22/09/2016 10:48, Jakub Zelenka wrote: > > > >> > >> Nope the point of the Bob's patch is to use graceful handling with > >> exception that can be easily checked by the json parser for e

Re: [PHP-DEV] HashDoS

On Thu, Sep 22, 2016 at 10:54 AM, Rowan Collins wrote: > On 22/09/2016 10:48, Jakub Zelenka wrote: > >> >> Nope the point of the Bob's patch is to use graceful handling with >> exception that can be easily checked by the json parser for example! See >> https://github.com/php/php-src/pull/1706 >>

Re: [PHP-DEV] HashDoS

On 22/09/2016 10:48, Jakub Zelenka wrote: Nope the point of the Bob's patch is to use graceful handling with exception that can be easily checked by the json parser for example! See https://github.com/php/php-src/pull/1706 Ah, I stand corrected, I hadn't seen that version referenced before. A

Re: [PHP-DEV] HashDoS

On Thu, Sep 22, 2016 at 10:06 AM, Rowan Collins wrote: > On 22/09/2016 08:52, Jakub Zelenka wrote: > >> I don't like the initial version of the patch that was causing fatal error >> for json_decode. That's not how json_decode should work. I think that Bob >> came up later with a better version th

Re: [PHP-DEV] HashDoS

On 22/09/2016 08:52, Jakub Zelenka wrote: I don't like the initial version of the patch that was causing fatal error for json_decode. That's not how json_decode should work. I think that Bob came up later with a better version that was using json recursion error. It might require a bit more work

Re: [PHP-DEV] HashDoS

On 22 Sep 2016 00:43, "Stanislav Malyshev" wrote: > > Hi! > > > As linked in my first reply, there was a previous discussion on the > > topic: http://markmail.org/message/ttbgcvdu4f7uymfb > > The collision-counting approach that Yasuo references is linked at the > > start of the first mail: https:

Re: [PHP-DEV] HashDoS

On 22 September 2016 00:23:16 BST, Stanislav Malyshev wrote: >In which case some limit like 1000 (just random number but can be >tested) would probably be OK. The question now is would it be enough to >block DoS? I.e. if we construct data to cause 999 collisions each time >to stay just under the

Re: [PHP-DEV] HashDoS

Hi! > As linked in my first reply, there was a previous discussion on the > topic: http://markmail.org/message/ttbgcvdu4f7uymfb > The collision-counting approach that Yasuo references is linked at the > start of the first mail: https://github.com/php/php-src/pull/1565 > > Collisions are counted d

Re: [PHP-DEV] HashDoS

Hi Stas, On Thu, Sep 22, 2016 at 7:47 AM, Stanislav Malyshev wrote: >> On Wed, Sep 21, 2016 at 11:26 AM, Stanislav Malyshev >> wrote: >>> I think we are better to limit max collisions. I'm +1 for Nikita's proposal does this. >>> >>> Max collision per what? How much would be the limit?

Re: [PHP-DEV] HashDoS

On Thu, Sep 22, 2016 at 1:07 AM, Stanislav Malyshev wrote: > Hi! > > >> Lets [try] to quantify the probability of reaching the collision limit C > > with a hashtable of size N and assuming a random hash distribution. The > > upper bound for this should be (N choose C) * (1/N)^(C-1), with > (1/N)^

Re: [PHP-DEV] HashDoS

Hi! > I'm not so sure of this now - unless, again, I misunderstand what we're > counting. It just occurred to me that I possibly do misunderstand what is counted - if the proposal is to count collisions per lookup. I rerun my scripts with that assumption and turns out the longest collision chain

Re: [PHP-DEV] HashDoS

Hi! >> Lets [try] to quantify the probability of reaching the collision limit C > with a hashtable of size N and assuming a random hash distribution. The > upper bound for this should be (N choose C) * (1/N)^(C-1), with (1/N)^(C-1) > being the probability that C elements hash to one value and (N o

Re: [PHP-DEV] HashDoS

Hi! > Hi Stas, > > On Wed, Sep 21, 2016 at 11:26 AM, Stanislav Malyshev > wrote: >> >>> I think we are better to limit max collisions. >>> I'm +1 for Nikita's proposal does this. >> >> Max collision per what? How much would be the limit? > > Collision by keys. Not sure I understand. What would

Re: [PHP-DEV] HashDoS

Hi Kiklas, On Wed, Sep 21, 2016 at 5:06 PM, Niklas Keller wrote: > 2016-09-21 8:54 GMT+02:00 Yasuo Ohgaki : >> >> Hi Stas, >> >> On Wed, Sep 21, 2016 at 11:26 AM, Stanislav Malyshev >> wrote: >> > >> >> I think we are better to limit max collisions. >> >> I'm +1 for Nikita's proposal does this.

Re: [PHP-DEV] HashDoS

On Wed, Sep 21, 2016 at 4:49 PM, Glenn Eggleton wrote: > This might be a bit off topic > > Given that you can set POST_REQUEST_SIZE in a production PHP application, > how likely is it really that an app will encounter a HashDos attack? > > From what I gather this will require MBs to GBs of da

Re: [PHP-DEV] HashDoS

This might be a bit off topic Given that you can set POST_REQUEST_SIZE in a production PHP application, how likely is it really that an app will encounter a HashDos attack? >From what I gather this will require MBs to GBs of data in order to cause a DoS. >From the web side, I think there are

Re: [PHP-DEV] HashDoS

On Wed, Sep 21, 2016 at 4:05 PM, Tom Worster wrote: > On 9/21/16 8:37 AM, Rowan Collins wrote: > >> On 21 September 2016 13:02:20 BST, Glenn Eggleton >> wrote: >> >>> What if we had some sort of configuration limit on collision length? >>> >> >> Previous discussions have come to the conclusion t

Re: [PHP-DEV] HashDoS

On 9/21/16 8:37 AM, Rowan Collins wrote: On 21 September 2016 13:02:20 BST, Glenn Eggleton wrote: What if we had some sort of configuration limit on collision length? Previous discussions have come to the conclusion that the difference between normal collision frequency and sufficient for a

Re: [PHP-DEV] HashDoS

On Wed, Sep 21, 2016 at 3:22 PM, Niklas Keller wrote: > 2016-09-21 14:37 GMT+02:00 Rowan Collins : > > > On 21 September 2016 13:02:20 BST, Glenn Eggleton > > wrote: > > >What if we had some sort of configuration limit on collision length? > > > > Previous discussions have come to the conclusion

Re: [PHP-DEV] HashDoS

On 9/20/16 10:25 PM, Stanislav Malyshev wrote: Note that to avoid problems with opcache we can only randomize on initial boot (even then synchronizing among different processes sharing opcache may be challenging). That means that the process would be running for extended time (at least days, in t

Re: [PHP-DEV] HashDoS

2016-09-21 14:37 GMT+02:00 Rowan Collins : > On 21 September 2016 13:02:20 BST, Glenn Eggleton > wrote: > >What if we had some sort of configuration limit on collision length? > > Previous discussions have come to the conclusion that the difference > between normal collision frequency and suffici

Re: [PHP-DEV] HashDoS

On 21 September 2016 13:02:20 BST, Glenn Eggleton wrote: >What if we had some sort of configuration limit on collision length? Previous discussions have come to the conclusion that the difference between normal collision frequency and sufficient for a DoS is so large that the only meaningful se

Re: [PHP-DEV] HashDoS

Hello, What if we had some sort of configuration limit on collision length? Would give most of us a viable way to prevent our apps from being attacked, and yet in the use cases where we require a higher limit we retain the ability to up the limit or disable it completely. Regards, Glenn On Wed,

Re: [PHP-DEV] HashDoS

2016-09-21 8:54 GMT+02:00 Yasuo Ohgaki : > Hi Stas, > > On Wed, Sep 21, 2016 at 11:26 AM, Stanislav Malyshev > wrote: > > > >> I think we are better to limit max collisions. > >> I'm +1 for Nikita's proposal does this. > > > > Max collision per what? How much would be the limit? > > Collision by

Re: [PHP-DEV] HashDoS

Hi Stas, On Wed, Sep 21, 2016 at 11:26 AM, Stanislav Malyshev wrote: > >> I think we are better to limit max collisions. >> I'm +1 for Nikita's proposal does this. > > Max collision per what? How much would be the limit? Collision by keys. It would be nice to have configurable limit like regex

Re: [PHP-DEV] HashDoS

Hi! > I think we are better to limit max collisions. > I'm +1 for Nikita's proposal does this. Max collision per what? How much would be the limit? -- Stas Malyshev smalys...@gmail.com -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] HashDoS

Hi! > - Do you think your proposed strategy can solve this problem entirely > without dropping djb3? > - Would randomization still help as a defense-in-depth? Note that to avoid problems with opcache we can only randomize on initial boot (even then synchronizing among different processes sharing

Re: [PHP-DEV] HashDoS

On Sat, Sep 17, 2016 at 6:35 PM, Yasuo Ohgaki wrote: > Hi all, > > On Sat, Sep 17, 2016 at 5:13 PM, Stanislav Malyshev > wrote: > >> Significant degradation? > >> > >> SipHash 1-3 is almost as fast as HashDoS-vulnerable hash > >> functions: https://github.com/funny-falcon/funny_hash > > > > I se

Re: [PHP-DEV] HashDoS

On 9/16/16 1:59 AM, Thomas Hruska wrote: If anyone wants a VERY rough estimate of relative performance degradation as a result of switching to SipHash, here's a somewhat naive C++ implementation of a similar data structure to that found in PHP: https://github.com/cubiclesoft/cross-platform-cpp

Re: [PHP-DEV] HashDoS

Hi all, On Sat, Sep 17, 2016 at 5:13 PM, Stanislav Malyshev wrote: >> Significant degradation? >> >> SipHash 1-3 is almost as fast as HashDoS-vulnerable hash >> functions: https://github.com/funny-falcon/funny_hash > > I see on this link comparison to Murmur3 - but that's not the function > we ar

Re: [PHP-DEV] HashDoS

Hi! > Significant degradation? > > SipHash 1-3 is almost as fast as HashDoS-vulnerable hash > functions: https://github.com/funny-falcon/funny_hash I see on this link comparison to Murmur3 - but that's not the function we are using. Is there a comparison to PHP one? -- Stas Malyshev smalys...@

Re: [PHP-DEV] HashDoS

Significant degradation? SipHash 1-3 is almost as fast as HashDoS-vulnerable hash functions: https://github.com/funny-falcon/funny_hash Scott Arciszewski Chief Development Officer Paragon Initiative Enterprises On Fri, Sep 16, 2016 at 1:59 AM, Thomas Hruska wrote: > On

Re: [PHP-DEV] HashDoS

On Fri, Sep 16, 2016 at 7:59 AM, Thomas Hruska wrote: > On 9/15/2016 5:20 PM, Stanislav Malyshev wrote: > >> Hi! >> >> On 9/15/16 11:48 AM, Scott Arciszewski wrote: >> >>> Would the Internals team be open to discussing mitigating HashDoS in a >>> future version of PHP? i.e. everywhere, even for j

Re: [PHP-DEV] HashDoS

On 9/15/2016 5:20 PM, Stanislav Malyshev wrote: Hi! On 9/15/16 11:48 AM, Scott Arciszewski wrote: Would the Internals team be open to discussing mitigating HashDoS in a future version of PHP? i.e. everywhere, even for json_decode() and friends, by fixing the problem rather than capping the maxi

Re: [PHP-DEV] HashDoS

Hi! On 9/15/16 11:48 AM, Scott Arciszewski wrote: > Would the Internals team be open to discussing mitigating HashDoS in a > future version of PHP? i.e. everywhere, even for json_decode() and friends, > by fixing the problem rather than capping the maximum number of input > parameters and hoping i

Re: [PHP-DEV] HashDoS

Hi Nikita, On Fri, Sep 16, 2016 at 3:56 AM, Nikita Popov wrote: > > Previous discussion on the topic: > http://markmail.org/message/ttbgcvdu4f7uymfb Your proposal is mandatory, IMHO. Let's implement it ASAP. Regards, -- Yasuo Ohgaki yohg...@ohgaki.net -- PHP Internals - PHP Runtime Developme

Re: [PHP-DEV] HashDoS

On Thu, Sep 15, 2016 at 8:48 PM, Scott Arciszewski wrote: > Would the Internals team be open to discussing mitigating HashDoS in a > future version of PHP? i.e. everywhere, even for json_decode() and friends, > by fixing the problem rather than capping the maximum number of input > parameters and

Re: [PHP-DEV] HashDos protection

On Dec 1, 2015 4:50 PM, "Dmitry Stogov" wrote: > > I think only big arrays coming from external sources should be checked. I tend to agree here. We discussed it with Remote last week. I was trying to explain why having a crafted hash function for inputs may be better and safer. That includes ge

Re: [PHP-DEV] HashDos protection

On Tue, Dec 1, 2015 at 3:48 PM, Nikita Popov wrote: > On Tue, Dec 1, 2015 at 10:50 AM, Dmitry Stogov wrote: > >> Hi Nikita, >> >> few notes: >> >> It looks like the patch messes the attempt of catching the problem (few >> lines related to zend_hash_find_bucket changes) with optimization to >> co

Re: [PHP-DEV] HashDos protection

On Tue, Dec 1, 2015 at 10:50 AM, Dmitry Stogov wrote: > Hi Nikita, > > few notes: > > It looks like the patch messes the attempt of catching the problem (few > lines related to zend_hash_find_bucket changes) with optimization to > compensate collision check overhead (everything else). I think it'

Re: [PHP-DEV] HashDos protection

On 1 December 2015 at 09:50, Dmitry Stogov wrote: > Hi Nikita, > > few notes: > > It looks like the patch messes the attempt of catching the problem (few > lines related to zend_hash_find_bucket changes) with optimization to > compensate collision check overhead (everything else). I think it's be

Re: [PHP-DEV] HashDos protection

Hi Nikita, few notes: It looks like the patch messes the attempt of catching the problem (few lines related to zend_hash_find_bucket changes) with optimization to compensate collision check overhead (everything else). I think it's better to separate these parts. Introduction of zend_hash_add_or_

Re: [PHP-DEV] HashDos protection

Le 01/12/2015 02:28, Anatol Belski a écrit : >From what I was testing, the configuration is not absolutely necessary. My first thinking, seeing this hardcoded limit was also, "we need to make it an new ini option". So I run a few tests. Especially the Anatol one (a bit optimized to get more uni

RE: [PHP-DEV] HashDos protection

Hi Stas, > -Original Message- > From: Stanislav Malyshev [mailto:smalys...@gmail.com] > Sent: Tuesday, December 1, 2015 12:21 AM > To: Nikita Popov ; PHP internals > ; Anatol Belski ; Remi Collet > > Subject: Re: [PHP-DEV] HashDos protection > > Hi

Re: [PHP-DEV] HashDos protection

Hi! > To fix the HashDos vulnerability for *all* cases (rather than just GET/POST > parsing), I propose to introduce collision counting during hashtable > insertion operations. This will throw a fatal error if the number of > collisions during an insertion operation exceed a certain threshold. >

Re: [PHP-DEV] HashDos protection

> 1. (Self-balancing binary trees). The idea here is that a balanced binary > tree has worst-case insertion time O(log N), while the linked list we > normally use has worst-case insertion time O(N). This means that the > worst-case execution time for N insertions into a hashtable goes from > O(N^2)

RE: [PHP-DEV] HashDos protection

Hi Thomas, > -Original Message- > From: Thomas Hruska [mailto:thru...@cubiclesoft.com] > Sent: Saturday, November 28, 2015 12:23 AM > To: Yasuo Ohgaki > Cc: PHP internals > Subject: Re: [PHP-DEV] HashDos protection > > On 11/27/2015 2:21 PM, Yasuo Ohg

Re: [PHP-DEV] HashDos protection

On Sat, Nov 28, 2015 at 12:22 AM, Thomas Hruska wrote: > On 11/27/2015 2:21 PM, Yasuo Ohgaki wrote: > >> Hi Thomas, >> >> In practice, we wouldn't have problems with max number of collisions. >> > > Is CLI going to be or can CLI be excluded from max collisions? After > thinking about it for a lo

RE: [PHP-DEV] HashDos protection

Hi Nikita, > -Original Message- > From: Nikita Popov [mailto:nikita@gmail.com] > Sent: Thursday, November 26, 2015 6:25 PM > To: PHP internals ; Anatol Belski > ; Remi Collet > Subject: [PHP-DEV] HashDos protection > > Hi internals! > > This mail turned out to be rather long, so I'l

Re: [PHP-DEV] HashDos protection

On 11/27/2015 2:21 PM, Yasuo Ohgaki wrote: Hi Thomas, In practice, we wouldn't have problems with max number of collisions. Is CLI going to be or can CLI be excluded from max collisions? After thinking about it for a long while, that's my only area of concern here. SAPI can (fatal) error t

Re: [PHP-DEV] HashDos protection

Hi Thomas, On Sat, Nov 28, 2015 at 2:07 AM, Thomas Hruska wrote: > I don't know if anyone has suggested this before, but why not have a > function that application developers can call to switch hash modes and > support multiple hash modes in the core? > > I've always viewed 'max_input_vars' as an

Re: [PHP-DEV] HashDos protection

Hi, On Thu, Nov 26, 2015 at 5:24 PM, Nikita Popov wrote: > > > What are your thoughts on this? > > First of all, thanks a lot for looking into it! That's great! I think that it's all cool except the fact that json_decode would result in fatal error. I don't think that json_decode should kill th

Re: [PHP-DEV] HashDos protection

On 11/27/2015 9:12 AM, Nikita Popov wrote: On Thu, Nov 26, 2015 at 8:35 PM, Niklas Keller wrote: Would this be a catchable Error (implementing Throwable) or a real fatal? Having a real fatal could lead to a DOS in Aerys as you'd be able to crash workers with carefully crafted input variables t

Re: [PHP-DEV] HashDos protection

On Thu, Nov 26, 2015 at 8:35 PM, Niklas Keller wrote: > Would this be a catchable Error (implementing Throwable) or a real fatal? > Having a real fatal could lead to a DOS in Aerys as you'd be able to crash > workers with carefully crafted input variables then. > It would be a real fatal error.

Re: [PHP-DEV] HashDos protection

Currently it's not catchable, that's my main concern. If it's catchable, it's not that much of a problem. Regards, Niklas 2015-11-27 10:05 GMT+01:00 Yasuo Ohgaki : > Hi Nikita, > > On Fri, Nov 27, 2015 at 2:24 AM, Nikita Popov > wrote: > > What are your thoughts on this? > > Great! This is exac

Re: [PHP-DEV] HashDos protection

Hi Nikita, On Fri, Nov 27, 2015 at 2:24 AM, Nikita Popov wrote: > What are your thoughts on this? Great! This is exactly what I was thinking. I prefer collision counting rather than slower hash function. Hardcoded collision max (1000) seems ok for me. Catchable fatal error, at least E_RECOVERAB

Re: [PHP-DEV] HashDos protection

On Thu, 26 Nov 2015 at 17:25 Nikita Popov wrote: > This will throw a fatal error if the number of > collisions during an insertion operation exceed a certain threshold. > To me this feels like it's just moving a DoS vector from one place to another. As Niklas already pointed out, he is directly

Re: [PHP-DEV] HashDos protection

> > 3. (Fatal error on many collisions). While the two previous options try to > ensure that hashtables stay efficient regardless of the used keys, the last > option aims to simply detect malicious array keys and abort the script in > such a case. > > This is done by counting the number of collisio