Hi Yasuo, > -----Original Message----- > From: Yasuo Ohgaki [mailto:yohg...@ohgaki.net] > Sent: Thursday, July 28, 2016 8:43 AM > To: Anatol Belski <a...@php.net> > Cc: php-...@lists.php.net > Subject: Re: [PHP-CVS] com php-src: Support Unicode characters in the local > part of an e-mail address.: ext/filter/filter.c ext/filter/filter_private.h > ext/filter/logical_filters.c ext/filter/tests/058.phpt > > Hi Anatol, > > I'm not comfortable with complex regex.... > > http://stackstatus.net/post/147710624694/outage-postmortem-july-20-2016 > This attack pattern works with PCRE even it has backtrack//stack limit. > > > Try following ruby code. You'll see exponential execution time increase. > 80 chars mail address requires 421 sec on my Core i7 fedora. > We are using PCRE, but I'm not comfortable with complex ereg... > > n = 5; > while n < 12 do > s = "username@host"+".abcde"*n+"." > start = Time.now(); > p /\A([\w+\-].?)+@[a-z\d\-]+(\.[a-z]+)*\.[a-z]+\z/i =~ s > p s.length.to_s + ': ' + (Time.now() - start).to_s; > n += 1 > end
Anything I've tried based on your Ruby snippet doesn't show times over 200us, whether it fails or passes. i7 here as well. The lenth is limited to 320 bytes in both cases. In the post you've linked, it's about over 20000 bytes length, and about replacing, not matching. The non UTF-8 pattern doesn't look simple as well, the new one extends it. Do you have some snippet to reproduce an attack vector with this concrete patch? Regards Anatol > -- > Yasuo Ohgaki > yohg...@ohgaki.net > > > On Tue, Jul 19, 2016 at 5:44 PM, Yasuo Ohgaki <yohg...@ohgaki.net> wrote: > > Hi Anatol, > > > > Although I'm not certain, I worry about ReDoS risks with this complex regex. > > > > https://www.checkmarx.com/wp-content/uploads/2015/03/ReDoS- > Attacks.pdf > > > > How about to use much simpler regex or string filter before applying > > the complex regex to reduce ReDoS risk? > > > > -- > > Yasuo Ohgaki > > yohg...@ohgaki.net > > > > > > On Tue, Jul 19, 2016 at 4:52 AM, Anatol Belski <a...@php.net> wrote: > >> Commit: 8f4050709c833b9d42cd65349b7747f8e61d171f > >> Author: Leo Feyer <git...@contao.org> Thu, 15 Oct 2015 11:07:25 > +0200 > >> Committer: Anatol Belski <a...@php.net> Mon, 18 Jul 2016 21:52:35 > >> +0200 > >> Parents: 42260325a47480bc97f7dceb2ac5d8e8d7799928 > >> Branches: master > >> > >> Link: http://git.php.net/?p=php- > src.git;a=commitdiff;h=8f4050709c833b9d42cd65349b7747f8e61d171f > >> > >> Log: > >> Support Unicode characters in the local part of an e-mail address. > >> > >> See RFC 6531 (https://tools.ietf.org/html/rfc6531). > >> > >> Encode IDNA domains. > >> > >> Add the missing "Done" to the expected output. > >> > >> Add the FILTER_FLAG_EMAIL_RFC6531 flag. > >> > >> Fix the filter_var() call. > >> > >> Changed paths: > >> M ext/filter/filter.c > >> M ext/filter/filter_private.h > >> M ext/filter/logical_filters.c > >> A ext/filter/tests/058.phpt > >> > >> > >> Diff: > >> diff --git a/ext/filter/filter.c b/ext/filter/filter.c index > >> 2c8dde9..3d79a77 100644 > >> --- a/ext/filter/filter.c > >> +++ b/ext/filter/filter.c > >> @@ -286,6 +286,8 @@ PHP_MINIT_FUNCTION(filter) > >> > >> REGISTER_LONG_CONSTANT("FILTER_FLAG_HOSTNAME", > >> FILTER_FLAG_HOSTNAME, CONST_CS | CONST_PERSISTENT); > >> > >> + REGISTER_LONG_CONSTANT("FILTER_FLAG_EMAIL_RFC6531", > >> + FILTER_FLAG_EMAIL_RFC6531, CONST_CS | CONST_PERSISTENT); > >> + > >> sapi_register_input_filter(php_sapi_filter, > >> php_sapi_filter_init); > >> > >> return SUCCESS; > >> diff --git a/ext/filter/filter_private.h > >> b/ext/filter/filter_private.h index 2f9abc3..ffce723 100644 > >> --- a/ext/filter/filter_private.h > >> +++ b/ext/filter/filter_private.h > >> @@ -57,6 +57,8 @@ > >> > >> #define FILTER_FLAG_HOSTNAME 0x100000 > >> > >> +#define FILTER_FLAG_EMAIL_RFC6531 0x100000 > >> + > >> #define FILTER_VALIDATE_INT 0x0101 > >> #define FILTER_VALIDATE_BOOLEAN 0x0102 > >> #define FILTER_VALIDATE_FLOAT 0x0103 > >> diff --git a/ext/filter/logical_filters.c > >> b/ext/filter/logical_filters.c index 5fd057d..03d4795 100644 > >> --- a/ext/filter/logical_filters.c > >> +++ b/ext/filter/logical_filters.c > >> @@ -598,7 +598,12 @@ void > php_filter_validate_email(PHP_INPUT_FILTER_PARAM_DECL) /* {{{ */ > >> * Feel free to use and redistribute this code. But please keep > >> this > copyright notice. > >> * > >> */ > >> - const char regexp[] = "/^(?!(?:(?:\\x22?\\x5C[\\x00- > \\x7E]\\x22?)|(?:\\x22?[^\\x5C\\x22]\\x22?)){255,})(?!(?:(?:\\x22?\\x5C[\\x00- > \\x7E]\\x22?)|(?:\\x22?[^\\x5C\\x22]\\x22?)){65,}@)(?:(?:[\\x21\\x23- > \\x27\\x2A\\x2B\\x2D\\x2F-\\x39\\x3D\\x3F\\x5E-\\x7E]+)|(?:\\x22(?:[\\x01- > \\x08\\x0B\\x0C\\x0E-\\x1F\\x21\\x23-\\x5B\\x5D-\\x7F]|(?:\\x5C[\\x00- > \\x7F]))*\\x22))(?:\\.(?:(?:[\\x21\\x23-\\x27\\x2A\\x2B\\x2D\\x2F- > \\x39\\x3D\\x3F\\x5E-\\x7E]+)|(?:\\x22(?:[\\x01-\\x08\\x0B\\x0C\\x0E- > \\x1F\\x21\\x23-\\x5B\\x5D-\\x7F]|(?:\\x5C[\\x00- > \\x7F]))*\\x22)))*@(?:(?:(?!.*[^.]{64,})(?:(?:(?:xn--)?[a-z0-9]+(?:-+[a-z0- > 9]+)*\\.){1,126}){1,}(?:(?:[a-z][a-z0-9]*)|(?:(?:xn--)[a-z0-9]+))(?:-+[a-z0- > 9]+)*)|(?:\\[(?:(?:IPv6:(?:(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){7})|(?:(?!(?:.*[a-f0- > 9][:\\]]){7,})(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,5})?::(?:[a-f0-9]{1,4}(?::[a-f0- > 9]{1,4}){0,5})?)))|(?:(?:IPv6:(?:(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){5}:)|(?:(?!(?:.*[a- > f0-9]:){5,})(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,3})?::(?:[a-f0-9]{1,4}(?::[a-f0- > 9]{1,4}){0,3}:)?)))?(?:(?:25[0-5])|(?:2[0-4][0-9])|(?:1[0-9]{2})|(?:[1-9]?[0- > 9]))(?:\\.(?:(?:25[0-5])|(?:2[0-4][0-9])|(?:1[0-9]{2})|(?:[1-9]?[0- > 9]))){3}))\\]))$/iD"; > >> + if (flags & FILTER_FLAG_EMAIL_RFC6531) { > >> + const char regexp[] = "/^(?!(?:(?:\\x22?\\x5C[\\x00- > \\x7E]\\x22?)|(?:\\x22?[^\\x5C\\x22]\\x22?)){255,})(?!(?:(?:\\x22?\\x5C[\\x00- > \\x7E]\\x22?)|(?:\\x22?[^\\x5C\\x22]\\x22?)){65,}@)(?:(?:[\\x21\\x23- > \\x27\\x2A\\x2B\\x2D\\x2F-\\x39\\x3D\\x3F\\x5E- > \\x7E\\pL\\pN]+)|(?:\\x22(?:[\\x01-\\x08\\x0B\\x0C\\x0E-\\x1F\\x21\\x23- > \\x5B\\x5D-\\x7F\\pL\\pN]|(?:\\x5C[\\x00- > \\x7F]))*\\x22))(?:\\.(?:(?:[\\x21\\x23-\\x27\\x2A\\x2B\\x2D\\x2F- > \\x39\\x3D\\x3F\\x5E-\\x7E\\pL\\pN]+)|(?:\\x22(?:[\\x01- > \\x08\\x0B\\x0C\\x0E-\\x1F\\x21\\x23-\\x5B\\x5D- > \\x7F\\pL\\pN]|(?:\\x5C[\\x00-\\x7F]))*\\x22)))*@(?:(?:(?!.*[^.]{64,})(?:(?:(?:xn- > -)?[a-z0-9]+(?:-+[a-z0-9]+)*\\.){1,126}){1,}(?:(?:[a-z][a-z0-9]*)|(?:(?:xn--)[a-z0- > 9]+))(?:-+[a-z0-9]+)*)|(?:\\[(?:(?:IPv6:(?:(?:[a-f0-9]{1,4}(?::[a-f0- > 9]{1,4}){7})|(?:(?!(?:.*[a-f0-9][:\\]]){7,})(?:[a-f0-9]{1,4}(?::[a-f0- > 9]{1,4}){0,5})?::(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,5})?)))|(?:(?:IPv6:(?:(?:[a-f0- > 9]{1,4}(?::[a-f0-9]{1,4}){5}:)|(?:(?!(?:.*[a-f0-9]:){5,})(?:[a-f0-9]{1,4}(?::[a-f0- > 9]{1,4}){0,3})?::(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,3}:)?)))?(?:(?:25[0-5])|(?:2[0- > 4][0-9])|(?:1[0-9]{2})|(?:[1-9]?[0-9]))(?:\\.(?:(?:25[0-5])|(?:2[0-4][0-9])|(?:1[0- > 9]{2})|(?:[1-9]?[0-9]))){3}))\\]))$/iDu"; > >> + } else { > >> + const char regexp[] = "/^(?!(?:(?:\\x22?\\x5C[\\x00- > \\x7E]\\x22?)|(?:\\x22?[^\\x5C\\x22]\\x22?)){255,})(?!(?:(?:\\x22?\\x5C[\\x00- > \\x7E]\\x22?)|(?:\\x22?[^\\x5C\\x22]\\x22?)){65,}@)(?:(?:[\\x21\\x23- > \\x27\\x2A\\x2B\\x2D\\x2F-\\x39\\x3D\\x3F\\x5E-\\x7E]+)|(?:\\x22(?:[\\x01- > \\x08\\x0B\\x0C\\x0E-\\x1F\\x21\\x23-\\x5B\\x5D-\\x7F]|(?:\\x5C[\\x00- > \\x7F]))*\\x22))(?:\\.(?:(?:[\\x21\\x23-\\x27\\x2A\\x2B\\x2D\\x2F- > \\x39\\x3D\\x3F\\x5E-\\x7E]+)|(?:\\x22(?:[\\x01-\\x08\\x0B\\x0C\\x0E- > \\x1F\\x21\\x23-\\x5B\\x5D-\\x7F]|(?:\\x5C[\\x00- > \\x7F]))*\\x22)))*@(?:(?:(?!.*[^.]{64,})(?:(?:(?:xn--)?[a-z0-9]+(?:-+[a-z0- > 9]+)*\\.){1,126}){1,}(?:(?:[a-z][a-z0-9]*)|(?:(?:xn--)[a-z0-9]+))(?:-+[a-z0- > 9]+)*)|(?:\\[(?:(?:IPv6:(?:(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){7})|(?:(?!(?:.*[a-f0- > 9][:\\]]){7,})(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,5})?::(?:[a-f0-9]{1,4}(?::[a-f0- > 9]{1,4}){0,5})?)))|(?:(?:IPv6:(?:(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){5}:)|(?:(?!(?:.*[a- > f0-9]:){5,})(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,3})?::(?:[a-f0-9]{1,4}(?::[a-f0- > 9]{1,4}){0,3}:)?)))?(?:(?:25[0-5])|(?:2[0-4][0-9])|(?:1[0-9]{2})|(?:[1-9]?[0- > 9]))(?:\\.(?:(?:25[0-5])|(?:2[0-4][0-9])|(?:1[0-9]{2})|(?:[1-9]?[0- > 9]))){3}))\\]))$/iD"; > >> + } > >> + > >> pcre *re = NULL; > >> pcre_extra *pcre_extra = NULL; > >> int preg_options = 0; > >> diff --git a/ext/filter/tests/058.phpt b/ext/filter/tests/058.phpt > >> new file mode 100644 index 0000000..a10f844 > >> --- /dev/null > >> +++ b/ext/filter/tests/058.phpt > >> @@ -0,0 +1,57 @@ > >> +--TEST-- > >> +FILTER_VALIDATE_EMAIL unicode support > >> +(https://tools.ietf.org/html/rfc6531) > >> +--SKIPIF-- > >> +<?php if (!extension_loaded("filter")) die("skip"); ?> > >> +--FILE-- > >> +<?php > >> +$values = Array( > >> +'niceändsim...@example.com', > >> +'véry.çom...@example.com', > >> +'a.lîttle.lengthy.but.fiñe...@dept.example.com', > >> +'dîsposable.style.émail.with+sym...@example.com', > >> +'other.émail-with-d...@example.com', > >> +'üser@[IPv6:2001:db8:1ff::a0b:dbd0]', > >> +'"verî.uñusual.@.uñusual.com"@example.com', > >> +'"verî.(),:;<>[]\".VERÎ.\"verî@\ > >> +\"verî\".unüsual"@strange.example.com', > >> +'tés...@example.com', > >> +'tést.ch...@example.com', > >> +'tés...@xn--exmple-cua.com', > >> +'tés...@xn----zfa.xe', > >> +'tést@subexample.wizard', > >> +'tést@[255.255.255.255]', > >> +'tést@[IPv6:2001:0db8:85a3:08d3:1319:8a2e:0370:7344]', > >> +'tést@[IPv6:2001::7344]', > >> +'tést@[IPv6:1111:2222:3333:4444:5555:6666:255.255.255.255]', > >> +'tést+refere...@example.com', > >> +'üñîçøðé@example.com', > >> +'"üñîçøðé"@example.com', > >> +'DžǼ੧ఘⅧ⒇৪@example.com', > >> +); > >> +foreach ($values as $value) { > >> + var_dump(filter_var($value, FILTER_VALIDATE_EMAIL, > >> +FILTER_FLAG_EMAIL_RFC6531)); } echo "Done\n"; ?> > >> +--EXPECT-- > >> +string(26) "niceändsim...@example.com" > >> +string(25) "véry.çom...@example.com" > >> +string(44) "a.lîttle.lengthy.but.fiñe...@dept.example.com" > >> +string(48) "dîsposable.style.émail.with+sym...@example.com" > >> +string(34) "other.émail-with-d...@example.com" > >> +string(35) "üser@[IPv6:2001:db8:1ff::a0b:dbd0]" > >> +string(43) ""verî.uñusual.@.uñusual.com"@example.com" > >> +string(74) > ""verî.(),:;<>[]\".VERÎ.\"verî@\ \"verî\".unüsual"@strange.example.com" > >> +string(17) "tés...@example.com" > >> +string(23) "tést.ch...@example.com" > >> +string(24) "tés...@xn--exmple-cua.com" > >> +string(18) "tés...@xn----zfa.xe" > >> +string(23) "tést@subexample.wizard" > >> +string(23) "tést@[255.255.255.255]" > >> +string(52) "tést@[IPv6:2001:0db8:85a3:08d3:1319:8a2e:0370:7344]" > >> +string(23) "tést@[IPv6:2001::7344]" > >> +string(58) "tést@[IPv6:1111:2222:3333:4444:5555:6666:255.255.255.255]" > >> +string(27) "tést+refere...@example.com" > >> +string(26) "üñîçøðé@example.com" > >> +string(28) ""üñîçøðé"@example.com" > >> +string(31) "DžǼ੧ఘⅧ⒇৪@example.com" > >> +Done > >> > >> > >> -- > >> PHP CVS Mailing List (http://www.php.net/) To unsubscribe, visit: > >> http://www.php.net/unsub.php > >> > > -- > PHP CVS Mailing List (http://www.php.net/) To unsubscribe, visit: > http://www.php.net/unsub.php -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
