RE: [PHP-DEV] Re: Windows OpCache bug fix

[Anatol Belski]

Hi Dmitry,

> -Original Message-
> From: Dmitry Stogov [mailto:dmi...@zend.com]
> Sent: Friday, October 2, 2015 9:34 AM
> To: Anatol Belski 
> Cc: Matt Ficken ; Pierre Joye
> ; Anatoliy Belsky ; Laruence
> ; PHP Internals ; dmi...@php.net
> Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> 
> > Dmitry, I'd have a question to this
> > > Also. if we can't map SHM into desired address space, we may map it
> > > in
> > any
> > > other address and copy data into the process memory similar to
> > file-cache.
> > In randomized memory layout, even the base were available and
> > OpenFileMapping were worky, some parts of that memory were already
> taken.
> > How exactly it could be done, could you please give a couple of
> > pointers to this?
> 
> 
> If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
> MapViewOfFileEx(..., NULL).
> 
> 
> > Would the file cache be always required then?
> >
> 
> This is not necessary, but depends on implementation of course.
> 
Thanks for the advice. I was playing with this over these days. There are two 
usual cases where it possibly fails when reattaching ATM

- 
https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_win32.c#L151
 - the saved address is available but is not suitable
- 
https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_win32.c#L159
 - the actual MapViewOfFileEx case

An unrelated plain C test shows, that MapViewOfFileEx can possibly fail when 
called second time, too. Even with NULL or with another address as base. But 
even if it could map at a different base - the internal structures will 
probably have invalid addresses. So it looks like there's indeed no profit to 
do any retry once the actualy base address needed was failed to reattach to.

IMHO the scenario that were usable - in case it fails to reattach to the exact 
address, it has to switch to heap. Just for one request, it should get a heap 
allocated segment and then invalidate all the cache. That way we fulfill the 
following

- the request is served for sure
- the scripts are not cached indeed, so no divergence with the actual real cache

A heap fallback memory handler would be probably quite easy to implement. What 
do you think?

Apropos with the heap - it also looks like when PHP is used as module under 
mpm_winnt, all the cache could use heap instead of SHM. In that case, there is 
only one Apache process serving with many threads. Except one would want to 
share that cache outside Apache, using heap there could be a much simpler 
solution.

Regards

Anatol




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] clone and new language construct consistency

[Gauthier Delamarre]

Hi all,

I once suggested to some core developers to offer « clone » the same syntax as 
« new » has since 5.4.

The idea being to be able to write this kind of code:

$foo = new Foo;
$bar = (clone $foo)->someMethod();

It seemed that this would not be that difficult to implement, and I was told 
that a RFC will be created for that purpose.

As I can’t find any RFC for that, I was wondering whether I should create one 
myself or if I just didn’t find it :)

Thanks,

Gauthier



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [PHP-DEV] clone and new language construct consistency

[Nikita Popov]

On Mon, Oct 5, 2015 at 9:48 AM, Gauthier Delamarre 
wrote:

> Hi all,
>
> I once suggested to some core developers to offer « clone » the same
> syntax as « new » has since 5.4.
>
> The idea being to be able to write this kind of code:
>
> $foo = new Foo;
> $bar = (clone $foo)->someMethod();
>
> It seemed that this would not be that difficult to implement, and I was
> told that a RFC will be created for that purpose.
>
> As I can’t find any RFC for that, I was wondering whether I should create
> one myself or if I just didn’t find it :)
>

Support for this was part of the UVS RFC (
https://wiki.php.net/rfc/uniform_variable_syntax), so this will work in PHP
7 :)

Nikita

Re: [PHP-DEV] PHP 7.0's Unicode version incoherence (mbstring, intl, pcre)

[Côme Chilliet]

Le jeudi 1 octobre 2015, 15:19:20 Tom Worster a écrit :
> Do people here agree that PHP should have a *policy* of using a consistent
> Unicode version?

I agree with this, seems like a fair request.

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] PHP 7.0's Unicode version incoherence (mbstring, intl, pcre)

[Martin Keckeis]

Hello,

2015-10-01 21:19 GMT+02:00 Tom Worster :
>
> Do people here agree that PHP should have a *policy* of using a consistent
> Unicode version?
>
> This appears to be easy to accomplish for the moment. Moving to Unicode 8
> will be harder.
>

I agree with the policy -> good idea.

But I think there will be a lot of problems, when staying on Unicode 7.
Since Unicode 8 has the new emoji (colors) and that is used more and more.

Re: [PHP-DEV] Re: [RFC] Arrow Functions

[Rowan Collins]

Tom Worster wrote on 03/10/2015 21:33:
when the grammar starts with function(args), it seems the main 
difference from existing grammar is to make the curly braces when 
there's only one statement in the function block.


in other contexts i had the impression that things like

if (bool-expr) statement;

and similar were going out of fashion fast.


Yeah, I'm not sure I agree with this line in the RFC at all:

> It also avoids the use of |{}| which means that writing closures 
in-line is less painful.


Avoiding the *return* statement I'm right behind, but if the => is no 
longer acting like an operator (which with the function keyword in 
place, it's really not), some extra brackets around the expression seem 
like they'd improve readability.


What about moving the => into the function(...) block?

function($foo) => $x * $y

vs

function($foo => $x * $y)

Re: [PHP-DEV] Re: Windows OpCache bug fix

[Dmitry Stogov]

On Mon, Oct 5, 2015 at 10:29 AM, Anatol Belski 
wrote:

> Hi Dmitry,
>
> > -Original Message-
> > From: Dmitry Stogov [mailto:dmi...@zend.com]
> > Sent: Friday, October 2, 2015 9:34 AM
> > To: Anatol Belski 
> > Cc: Matt Ficken ; Pierre Joye
> > ; Anatoliy Belsky ; Laruence
> > ; PHP Internals ;
> dmi...@php.net
> > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> >
> > > Dmitry, I'd have a question to this
> > > > Also. if we can't map SHM into desired address space, we may map it
> > > > in
> > > any
> > > > other address and copy data into the process memory similar to
> > > file-cache.
> > > In randomized memory layout, even the base were available and
> > > OpenFileMapping were worky, some parts of that memory were already
> > taken.
> > > How exactly it could be done, could you please give a couple of
> > > pointers to this?
> >
> >
> > If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
> > MapViewOfFileEx(..., NULL).
> >
> >
> > > Would the file cache be always required then?
> > >
> >
> > This is not necessary, but depends on implementation of course.
> >
> Thanks for the advice. I was playing with this over these days. There are
> two usual cases where it possibly fails when reattaching ATM
>
> -
> https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_win32.c#L151
> - the saved address is available but is not suitable
> -
> https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_win32.c#L159
> - the actual MapViewOfFileEx case
>
> An unrelated plain C test shows, that MapViewOfFileEx can possibly fail
> when called second time, too. Even with NULL or with another address as
> base. But even if it could map at a different base - the internal
> structures will probably have invalid addresses.


right. we might need different code for zend_accle_hash access or convert
corresponding structures to PIC.
For opcdeos "invalid address" don't matter because we will copy them into
process memory and fix (like in file-cache).


> So it looks like there's indeed no profit to do any retry once the actualy
> base address needed was failed to reattach to.
>
> IMHO the scenario that were usable - in case it fails to reattach to the
> exact address, it has to switch to heap. Just for one request, it should
> get a heap allocated segment and then invalidate all the cache. That way we
> fulfill the following
>
> - the request is served for sure
> - the scripts are not cached indeed, so no divergence with the actual real
> cache
>
> A heap fallback memory handler would be probably quite easy to implement.
> What do you think?
>
> Apropos with the heap - it also looks like when PHP is used as module
> under mpm_winnt, all the cache could use heap instead of SHM. In that case,
> there is only one Apache process serving with many threads. Except one
> would want to share that cache outside Apache, using heap there could be a
> much simpler solution.
>

Heap based cache makes the same problems. It increase the memory usage and
doesn't provide cache consistency.
Just fall-back to file-cache may be better.

Actually, when I implemented file-cache I had a though about different
storage back-ends (e.g. sqlite or memcache).
This abstraction might make support for "improperly mapped SHM" quite easy.

Thanks. Dmitry.



>
> Regards
>
> Anatol
>
>
>
>

[PHP-DEV] Benchmark Results for PHP Master 2015-10-05

[lp_benchmark_robot]

Results for project php-src-nightly, build date 2015-10-05 05:13:19+03:00
commit: 746b37e2d551c89d25364ffefee19445651f93d6
revision_date:  2015-10-05 00:57:30+02:00
environment:Haswell-EP
cpu:Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 
2, LLC 45 MB
mem:128 GB
os: CentOS 7.1
kernel: Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release php-7.0.0beta3, with hash
1674bd9b151ff389fb8c9fc223bc6aafdd49ff2c from 2015-08-05 04:56:40+00:00

--
benchmark   relative   change since   change since
std_dev*   last run php-7.0.0beta3
--
:-)   Wordpress 4.2.2 cgi -T1  0.25%  0.28%  2.84%
:-|   Drupal 7.36 cgi -T1  0.94% -0.58%  1.58%
:-)   MediaWiki 1.23.9 cgi -T5000  0.24% -0.42%  3.27%
:-) bench.php cgi -T1  0.07% -0.06%  2.30%
:-(   micro_bench.php cgi -T1  0.02%  0.73% -3.39%
:-)mandelbrot.php cgi -T1  0.20% -0.01%  7.99%
--
Note: Benchmark results for Wordpress, Drupal, MediaWiki are measured in
fetches/second while all others are measured in seconds.
* Relative Standard Deviation (Standard Deviation/Average)

Our lab does a nightly source pull and build of the PHP project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.


-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

RE: [PHP-DEV] Re: Windows OpCache bug fix

[Anatol Belski]

> -Original Message-
> From: Dmitry Stogov [mailto:dmi...@zend.com]
> Sent: Monday, October 5, 2015 1:08 PM
> To: Anatol Belski 
> Cc: Matt Ficken ; Pierre Joye
> ; Laruence ; PHP Internals
> ; dmi...@php.net
> Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> 
> On Mon, Oct 5, 2015 at 10:29 AM, Anatol Belski 
> wrote:
> 
> > Hi Dmitry,
> >
> > > -Original Message-
> > > From: Dmitry Stogov [mailto:dmi...@zend.com]
> > > Sent: Friday, October 2, 2015 9:34 AM
> > > To: Anatol Belski 
> > > Cc: Matt Ficken ; Pierre Joye
> > > ; Anatoliy Belsky ; Laruence
> > > ; PHP Internals ;
> > dmi...@php.net
> > > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> > >
> > > > Dmitry, I'd have a question to this
> > > > > Also. if we can't map SHM into desired address space, we may map
> > > > > it in
> > > > any
> > > > > other address and copy data into the process memory similar to
> > > > file-cache.
> > > > In randomized memory layout, even the base were available and
> > > > OpenFileMapping were worky, some parts of that memory were already
> > > taken.
> > > > How exactly it could be done, could you please give a couple of
> > > > pointers to this?
> > >
> > >
> > > If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
> > > MapViewOfFileEx(..., NULL).
> > >
> > >
> > > > Would the file cache be always required then?
> > > >
> > >
> > > This is not necessary, but depends on implementation of course.
> > >
> > Thanks for the advice. I was playing with this over these days. There
> > are two usual cases where it possibly fails when reattaching ATM
> >
> > -
> > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
> > in32.c#L151
> > - the saved address is available but is not suitable
> > -
> > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
> > in32.c#L159
> > - the actual MapViewOfFileEx case
> >
> > An unrelated plain C test shows, that MapViewOfFileEx can possibly
> > fail when called second time, too. Even with NULL or with another
> > address as base. But even if it could map at a different base - the
> > internal structures will probably have invalid addresses.
> 
> 
> right. we might need different code for zend_accle_hash access or convert
> corresponding structures to PIC.
> For opcdeos "invalid address" don't matter because we will copy them into
> process memory and fix (like in file-cache).
> 
Ah, I have to study the file cache code then. But generally it sounds not like 
something that can be done offhand. I've also thought about other things like 
interned strings (maybe something else), not sure they're stored with the cache.

> 
> > So it looks like there's indeed no profit to do any retry once the
> > actualy base address needed was failed to reattach to.
> >
> > IMHO the scenario that were usable - in case it fails to reattach to
> > the exact address, it has to switch to heap. Just for one request, it
> > should get a heap allocated segment and then invalidate all the cache.
> > That way we fulfill the following
> >
> > - the request is served for sure
> > - the scripts are not cached indeed, so no divergence with the actual
> > real cache
> >
> > A heap fallback memory handler would be probably quite easy to implement.
> > What do you think?
> >
> > Apropos with the heap - it also looks like when PHP is used as module
> > under mpm_winnt, all the cache could use heap instead of SHM. In that
> > case, there is only one Apache process serving with many threads.
> > Except one would want to share that cache outside Apache, using heap
> > there could be a much simpler solution.
> >
> 
> Heap based cache makes the same problems. It increase the memory usage and
> doesn't provide cache consistency.
> Just fall-back to file-cache may be better.
Do you think something like this would suffice as file cache fallback 
https://gist.github.com/weltling/224001a468f04de13693  ? Though it'd still 
diverge from the "main" cache.

Actually in such case all the processes should switch to file cache? Just not 
sure how they all would negotiate that when no SHM is available (probably 
through files, or a separate shared chunk).

> 
> Actually, when I implemented file-cache I had a though about different storage
> back-ends (e.g. sqlite or memcache).
> This abstraction might make support for "improperly mapped SHM" quite easy.
> 
Yeah, that could be useful. Maybe also module based so one could even supply 
them separately, then an arbitrary storage mechanism could be provided. Fe like 
I've mentioned for mpm_winnt -if there's no intention to share the cache 
outside Apache, just one heap for all could be much simpler to avoid all that 
reattach mechanics.

Thanks.

Anatol


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: Windows OpCache bug fix

[Dmitry Stogov]

On Mon, Oct 5, 2015 at 3:51 PM, Anatol Belski  wrote:

>
>
> > -Original Message-
> > From: Dmitry Stogov [mailto:dmi...@zend.com]
> > Sent: Monday, October 5, 2015 1:08 PM
> > To: Anatol Belski 
> > Cc: Matt Ficken ; Pierre Joye
> > ; Laruence ; PHP Internals
> > ; dmi...@php.net
> > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> >
> > On Mon, Oct 5, 2015 at 10:29 AM, Anatol Belski 
> > wrote:
> >
> > > Hi Dmitry,
> > >
> > > > -Original Message-
> > > > From: Dmitry Stogov [mailto:dmi...@zend.com]
> > > > Sent: Friday, October 2, 2015 9:34 AM
> > > > To: Anatol Belski 
> > > > Cc: Matt Ficken ; Pierre Joye
> > > > ; Anatoliy Belsky ; Laruence
> > > > ; PHP Internals ;
> > > dmi...@php.net
> > > > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> > > >
> > > > > Dmitry, I'd have a question to this
> > > > > > Also. if we can't map SHM into desired address space, we may map
> > > > > > it in
> > > > > any
> > > > > > other address and copy data into the process memory similar to
> > > > > file-cache.
> > > > > In randomized memory layout, even the base were available and
> > > > > OpenFileMapping were worky, some parts of that memory were already
> > > > taken.
> > > > > How exactly it could be done, could you please give a couple of
> > > > > pointers to this?
> > > >
> > > >
> > > > If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
> > > > MapViewOfFileEx(..., NULL).
> > > >
> > > >
> > > > > Would the file cache be always required then?
> > > > >
> > > >
> > > > This is not necessary, but depends on implementation of course.
> > > >
> > > Thanks for the advice. I was playing with this over these days. There
> > > are two usual cases where it possibly fails when reattaching ATM
> > >
> > > -
> > > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
> > > in32.c#L151
> > > - the saved address is available but is not suitable
> > > -
> > > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
> > > in32.c#L159
> > > - the actual MapViewOfFileEx case
> > >
> > > An unrelated plain C test shows, that MapViewOfFileEx can possibly
> > > fail when called second time, too. Even with NULL or with another
> > > address as base. But even if it could map at a different base - the
> > > internal structures will probably have invalid addresses.
> >
> >
> > right. we might need different code for zend_accle_hash access or convert
> > corresponding structures to PIC.
> > For opcdeos "invalid address" don't matter because we will copy them into
> > process memory and fix (like in file-cache).
> >
> Ah, I have to study the file cache code then. But generally it sounds not
> like something that can be done offhand. I've also thought about other
> things like interned strings (maybe something else), not sure they're
> stored with the cache.
>
> >
> > > So it looks like there's indeed no profit to do any retry once the
> > > actualy base address needed was failed to reattach to.
> > >
> > > IMHO the scenario that were usable - in case it fails to reattach to
> > > the exact address, it has to switch to heap. Just for one request, it
> > > should get a heap allocated segment and then invalidate all the cache.
> > > That way we fulfill the following
> > >
> > > - the request is served for sure
> > > - the scripts are not cached indeed, so no divergence with the actual
> > > real cache
> > >
> > > A heap fallback memory handler would be probably quite easy to
> implement.
> > > What do you think?
> > >
> > > Apropos with the heap - it also looks like when PHP is used as module
> > > under mpm_winnt, all the cache could use heap instead of SHM. In that
> > > case, there is only one Apache process serving with many threads.
> > > Except one would want to share that cache outside Apache, using heap
> > > there could be a much simpler solution.
> > >
> >
> > Heap based cache makes the same problems. It increase the memory usage
> and
> > doesn't provide cache consistency.
> > Just fall-back to file-cache may be better.
> Do you think something like this would suffice as file cache fallback
> https://gist.github.com/weltling/224001a468f04de13693  ? Though it'd
> still diverge from the "main" cache.
>

I think we should enable file-cache automatically, but we can set
ZCG(accel_directives).file_cache_only
if the file_cache already enabled.


>
> Actually in such case all the processes should switch to file cache?


No. Only the processes that weren't be able to attach to SHM.


> Just not sure how they all would negotiate that when no SHM is available
> (probably through files, or a separate shared chunk).
>

yeah. Processes that use file-cache-only won't be able to negotiate SHM
cache. :(


>
> >
> > Actually, when I implemented file-cache I had a though about different
> storage
> > back-ends (e.g. sqlite or memcache).
> > This abstraction might make support for "improperly mapped SHM" quite
> easy.
> >
> Yeah, that could be useful. Maybe also module based so one c

[PHP-DEV] Re: PHP 7.1 - should we add a random_str() function?

[Scott Arciszewski]

On Fri, Oct 2, 2015 at 5:10 PM, Tom Worster  wrote:
> I screwed up sending this earlier. Sorry if you get this twice.
>
> On 9/30/15 12:15 PM, Scott Arciszewski wrote:
>> I think random_bytes() and random_int() are great; they provide a
>> much-needed building block in PHP 7.0. However, I do worry a bit that
>> the most common use for random_int() (generating a random string of a
>> fixed length with a given character set) will be reinvented over and
>> over again, and rarely consistently.
>
> On one had I agree that it's a common use and put a method in Yii2's
> Security component for it (albeit less general then your proposal). But I'm
> not sure the motive you gave is sufficient to put it in PHP core.
>
> We should be less concerned about people reinventing it over and over than
> people getting it wrong. The SO answer you referenced expresses exactly this
> concern. This was the motive for the new random functions and the password
> hash functions. It's a good argument.
>
>
>> I would propose a random_str() function that behaves similar to this
>> userland snippet: http://stackoverflow.com/a/32870871/2224584
>>
>> Function prototype:
>>
>>> string random_str( int $length, string $charset)
>>
>> Would return a string or throw an Error|Exception (e.g. invalid input
>> parameters, or the operating system's CSPRNG begins to melt).
>
> If the problem is poor algorithms generating random strings that get "used
> for anything remotely analogous to a password" then I think this is not
> enough to be a solution. I think a class is needed that can do more
> including:
>
> - Unicode characters. The $random_str .= $charset[$r]; line in the snippet
> you referenced implies a rather parochial tacit assumption. Passwords aren't
> always limited to Basic Latin.
>
> - Constraints such as: exclude easily confusable characters such as 0 and O,
> at least one digit, pronounceable, and things like that
>
> - Choosing from a set of words (Diceware)
>
> From my own interactions with others, I know that people who shouldn't do in
> fact attempt to implement this kind of stuff. And that's to be expected
> because PHP apps very often need it.
>
>
>
>> I can write up an RFC for this, with a patch targeting 7.1, if anyone
>> is interested in it.
>>
>> Scott Arciszewski
>> Chief Development Officer
>> Paragon Initiative Enterprises 
>>
>

You mentioned diceware. Incidentally,
https://paragonie.com/blog/2015/07/common-uses-for-csprngs-cryptographically-secure-pseudo-random-number-generators#diceware

Some problems (i.e. random_int) should be fixed at a language level.
Others, through education. In the end, this might be an education
issue.

Scott Arciszewski
Chief Development Officer
Paragon Initiative Enterprises 

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

RE: [PHP-DEV] Re: Windows OpCache bug fix

[Anatol Belski]

> -Original Message-
> From: Dmitry Stogov [mailto:dmi...@zend.com]
> Sent: Monday, October 5, 2015 3:31 PM
> To: Anatol Belski 
> Cc: Matt Ficken ; Pierre Joye
> ; Laruence ; PHP Internals
> ; dmi...@php.net
> Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> 
> > > > > Subject: Re: [PHP-DEV] Re: Windows OpCache bug fix
> > > > >
> > > > > > Dmitry, I'd have a question to this
> > > > > > > Also. if we can't map SHM into desired address space, we may
> > > > > > > map it in
> > > > > > any
> > > > > > > other address and copy data into the process memory similar
> > > > > > > to
> > > > > > file-cache.
> > > > > > In randomized memory layout, even the base were available and
> > > > > > OpenFileMapping were worky, some parts of that memory were
> > > > > > already
> > > > > taken.
> > > > > > How exactly it could be done, could you please give a couple
> > > > > > of pointers to this?
> > > > >
> > > > >
> > > > > If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
> > > > > MapViewOfFileEx(..., NULL).
> > > > >
> > > > >
> > > > > > Would the file cache be always required then?
> > > > > >
> > > > >
> > > > > This is not necessary, but depends on implementation of course.
> > > > >
> > > > Thanks for the advice. I was playing with this over these days.
> > > > There are two usual cases where it possibly fails when reattaching
> > > > ATM
> > > >
> > > > -
> > > > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_all
> > > > oc_w
> > > > in32.c#L151
> > > > - the saved address is available but is not suitable
> > > > -
> > > > https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_all
> > > > oc_w
> > > > in32.c#L159
> > > > - the actual MapViewOfFileEx case
> > > >
> > > > An unrelated plain C test shows, that MapViewOfFileEx can possibly
> > > > fail when called second time, too. Even with NULL or with another
> > > > address as base. But even if it could map at a different base -
> > > > the internal structures will probably have invalid addresses.
> > >
> > >
> > > right. we might need different code for zend_accle_hash access or
> > > convert corresponding structures to PIC.
> > > For opcdeos "invalid address" don't matter because we will copy them
> > > into process memory and fix (like in file-cache).
> > >
> > Ah, I have to study the file cache code then. But generally it sounds
> > not like something that can be done offhand. I've also thought about
> > other things like interned strings (maybe something else), not sure
> > they're stored with the cache.
> >
> > >
> > > > So it looks like there's indeed no profit to do any retry once the
> > > > actualy base address needed was failed to reattach to.
> > > >
> > > > IMHO the scenario that were usable - in case it fails to reattach
> > > > to the exact address, it has to switch to heap. Just for one
> > > > request, it should get a heap allocated segment and then invalidate all 
> > > > the
> cache.
> > > > That way we fulfill the following
> > > >
> > > > - the request is served for sure
> > > > - the scripts are not cached indeed, so no divergence with the
> > > > actual real cache
> > > >
> > > > A heap fallback memory handler would be probably quite easy to
> > implement.
> > > > What do you think?
> > > >
> > > > Apropos with the heap - it also looks like when PHP is used as
> > > > module under mpm_winnt, all the cache could use heap instead of
> > > > SHM. In that case, there is only one Apache process serving with many
> threads.
> > > > Except one would want to share that cache outside Apache, using
> > > > heap there could be a much simpler solution.
> > > >
> > >
> > > Heap based cache makes the same problems. It increase the memory
> > > usage
> > and
> > > doesn't provide cache consistency.
> > > Just fall-back to file-cache may be better.
> > Do you think something like this would suffice as file cache fallback
> > https://gist.github.com/weltling/224001a468f04de13693  ? Though it'd
> > still diverge from the "main" cache.
> >
> 
> I think we should enable file-cache automatically, but we can set
> ZCG(accel_directives).file_cache_only
> if the file_cache already enabled.
> 
I've revoked the approach https://gist.github.com/weltling/69bd1e47dc15273edde5 
, also added enforcement per request (was missing in the previous version). Or 
did you mean "we should NOT enable file cache automatically"? Can be easy 
changed ofc. IMHO one can enforce automatically, careful programmers do check 
error logs :)

> 
> >
> > Actually in such case all the processes should switch to file cache?
> 
> 
> No. Only the processes that weren't be able to attach to SHM.
> 
> 
> > Just not sure how they all would negotiate that when no SHM is
> > available (probably through files, or a separate shared chunk).
> >
> 
> yeah. Processes that use file-cache-only won't be able to negotiate SHM cache.
> :(
> 
ACK, so basically it is the same principle Matt suggested with sidestep cache. 
I could imagine synchronizing all the p

[PHP-DEV] Re: PHP 7.1 - should we add a random_str() function?

[Tom Worster]

On 10/5/15, 11:34 AM, "Scott Arciszewski"  wrote:
>
>You mentioned diceware. Incidentally,
>https://paragonie.com/blog/2015/07/common-uses-for-csprngs-cryptographical
>ly-secure-pseudo-random-number-generators#diceware
>
>Some problems (i.e. random_int) should be fixed at a language level.
>Others, through education. In the end, this might be an education
>issue.
>
>Scott Arciszewski
>Chief Development Officer
>Paragon Initiative Enterprises 

Hi Scott,

I don't think a random string generator or the other stuff I mentioned
should be in standard PHP API. If PHP wants to help with this sort of
thing (I think it would be valuable) the better way is a new ext with a
class that provides some utility methods and such, as I described before.

With respect to your code, if you assume an opcode cache, a trait is a
decent way to save a large lookup table, such as the Diceware word list,
as a PHP array. Examples in the two data trait classes here:
https://github.com/tom--/precis Better still, compile it into PHP.

Your random string generator appears to assume either that the $charset
string is limited to ASCII7 (Basic Latin) or that the PHP file is latin
encoded. I don't think it's a safe assumption to make in 2015.

Tom



-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] PHP 7.0's Unicode version incoherence (mbstring, intl, pcre)

[Tom Worster]

On 10/5/15 5:20 AM, Martin Keckeis wrote:

Hello,

2015-10-01 21:19 GMT+02:00 Tom Worster :


Do people here agree that PHP should have a *policy* of using a consistent
Unicode version?

This appears to be easy to accomplish for the moment. Moving to Unicode 8
will be harder.



I agree with the policy -> good idea.

But I think there will be a lot of problems, when staying on Unicode 7.
Since Unicode 8 has the new emoji (colors) and that is used more and more.


The new emoji are neat and people can and should use them. They just 
can't expect intl or preg to recognize them until some release after 7.0.0.


Using Unicode 8 uniformly involves moving to PCRE2, which doesn't seem 
easy. It's a big API change for little functional gain.


I'd like to see PCRE2 implement a bit more of TR18 but I doubt it's a 
priority. ICU regex as the next big thing in intl is an interesting 
idea. I've no idea how hard it is.

http://unicode.org/reports/tr18/

So Unicode 8 depends on what PHP does about regex going forwards. I 
asked about this on Sep 24 but got no response.

http://www.serverphorums.com/read.php?7,1303221

Tom

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [RFC] [DRAFT]: Consistent callables

[Rowan Collins]

Dan Ackroyd wrote on 29/09/2015 15:25:

Hello internals,

Here is a draft RFC to make the callable type in PHP be consistent and
have much fewer surprises.

https://wiki.php.net/rfc/consistent_callables


To break the silence: I really like the sentiment behind this, and it 
seems very thorough. I've been caught out several times by the fact that 
this appears to work:


$callback = [self, 'some_method'];

Because of the legacy "feature" of treating undefined constants as 
strings (which IMHO should be deprecated anyway, and removed in 8.0), it 
becomes this:


$callback = ['self', 'some_method'];

... and the 'self' is only evaluated when you come to run it, at which 
point it will either fatal error, or do something completely irrelevant, 
depending if that name happens to exist in the class context it finds 
itself in.



The only part I have a slight concern over is the behaviour of 
private/protected methods, but if some form of constructor is available 
to neatly wrap them into closures, I think it's acceptable to require 
its use.


The way I see it is that currently is_callable($foo) means "is $foo 
callable in the current context", whereas the proposal makes it "is $foo 
a valid value of type callable".



My only other thought is that deprecation should happen in 7.1, not 
"7.last", because a) we should give people as much warning as possible 
(as soon as the replacement functionality is available to move to) 
rather than leaving until the last minute; and b) we won't necessarily 
know that a particular release is 7.last, e.g. the decision against 
releasing a 5.7 happened 6 months after the release of 5.6.0


Regards,
--
Rowan Collins
[IMSoP]


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

RE: [PHP-DEV] Re: Windows OpCache bug fix

[Eric Stenson]

>From: Dmitry Stogov [mailto:dmi...@zend.com] 
>
>> On Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken 
>> wrote:
>>
>>> Pierre Joy wrote:
>>> And what wincache does. It is slower but the request is served.
>>
>> WinCache (file cache) if it can't reattach, creates a new shared mem 
>> file
>
> I'm not sure how WinCache works, but opcache already implements most 
> necessary functionality. 

Sorry for getting to the party late, but let me clarify what WinCache does:

1. For the opcode cache, if WinCache can't map to the same address, it falls 
back to using process-local (heap) memory for the opcode arrays.  This is 
because the elements in the opcode arrays contain absolute pointers, and we 
don't want to spend cycles (a) converting all pointers to offsets when copying 
opcode arrays into shared memory, and then (b) converting all offsets back to 
absolute addresses when copying the opcode array back out of shared memory.  
WinCache simply gets the pointer to the opcode array in shared memory and 
returns it, unmodified, to the PHP Core for execution.

When WinCache falls back to process-local memory, it does NOT create a second 
memory mapped segment, and does not chew additional cross-process resources.

2. WinCache's file cache does not map to an identical address in all processes. 
 We simply map it to *any* address (using NULL for the suggested address to 
MapViewOfFileEx), and then use relative offsets in the cache structures for all 
entries in shared memory.

3. WinCache's user cache and session cache are similar to the file cache, and 
do not have to map to the same address in all processes. Cache entries use 
relative offsets to locate cache entries, and the values do not contain 
absolute addresses. 

Finally, For PHP7, WinCache has removed its opcode cache.  We assume that 
Windows consumers of PHP7 will use Zend Opcache for opcode caching.

Thx!
--E. 

Re: [PHP-DEV] RFC proposal.

[Rowan Collins]

On 4 October 2015 23:10:25 BST, somebody anon 
 wrote:
>Hi PHP community. I am a developer with few years experience in PHP and
>now
>want to propose some addition to default "filesystem" extension.
>There's
>money_format and number_format functions to format money and number
>values
>in human-readable format, but there is not any "size_format" function
>that
>does the same - reformat an integer that holds size in bytes in
>well-formed
>string representing size in one of "big" units like MB or GB. So I can
>implement function to do this with following interface:
>> size_format($format, $bytes, $unit = null)
>that will format size like printf with taking in accoun using unit and
>multiplier (1k or 1024).
>So, I wait for responses. Thanks for attention.

An interesting idea. Do you have any idea of the values $format and $unit would 
accept? Would it handle both decimal-based GB etc and binary-based GiB etc [1]?

What about other units that can be scaled in the same way, like metres? Would 
it make more sense to make it a unit-formatting function or library - and if 
so, does it need to be in core, or could it just be a lightweight Composer 
dependency people could grab when they needed it? 

[1] https://en.m.wikipedia.org/wiki/Binary_prefix

Regards,
-- 
Rowan Collins
[IMSoP]


-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: Windows OpCache bug fix

[Matt Ficken]

Ultimately if the 2nd OpCache is shared in a new SHM, in-process in a new
SHM or in-process Heap, memory will increase, scripts have to be recompiled
and there are consistency issues. File-cache will avoid recompiling
scripts, but unless OpCache is disabled, there needs to be a 2nd OpCache
stored somewhere.

For the stat, reset and revalidate consistency issues, to build off
Anatol's idea:  If can't reattach to base address, then reattach to any
address and create a 2nd OpCache in Heap, BUT keep the SHM open so that the
stat fields(fe ZCSG(hits)) are still accessible. Those fields aren't
pointers so they should work from whatever base address MapViewOfFileEx()
provides, if not, could fork ZCSG into 2 structs (somehow reuse existing
stat fields on existing SHM).

opcache_get_status() in any/all processes would read those fields from SHM.


Since there would be 2+ processes with an OpCache (the original in SHM, and
sidestep OpCaches on the heaps of those processes), can't use a simple flag
(would be cleared by 2nd process, before 3rd got it).  Could add
ZCSG(opcache_count) field and increment every time an OpCache gets created
(and decrement on process exit). Simple, but not using
zend_shared_alloc_lock() avoids creating new locking issues.

Add a new ZCSG(opcache_revalidate_count) in that struct that is set to
ZCSG(opcache_count) anytime opcache_reset() or opcache_invalidate() called
or if persistent_compile_file() finds a changed script.

In process with side-step OpCache,  in persistent_compile_file()  (assuming
opcache.validate_timestamps && revalidate_freq>0) will check if
ZCSG(opcache_revalidate_count) > 0 and will then decrement the count and
reload scripts from file-cache (where it checks opcache.revalidate_freq).
And then wrap this all in some `#ifdef ZEND_WIN32`s


Would still have extra memory usage. When I have a process that can't
reattach I usually get several, so having an in-process OpCache(SHM or
Heap) may lead to more memory usage than a shared SHM, but having 2 SHMs in
a process would mess up the nice existing code.

Don't have to recompile scripts(quickly loads from file-cache) and fixes
the 3 consistency issues, and would keep serving up requests without
performance loss.

Yes, temporary incomplete solutions often get forgotten and not replaced
later with a better complete solution, but in this case we're motivated try
to fully fix this whole issue and if that doesn't work, at least build a
more complete solution, but that takes time (known issue for already a few
years), we should do something that works for now. This is more complex
than a bugfix that creates a 2nd SHM OpCache. Can we do this now? What is
still possible for 7.0? 7.0.1?



On Mon, Oct 5, 2015 at 11:49 AM, Eric Stenson 
wrote:

> >From: Dmitry Stogov [mailto:dmi...@zend.com]
> >
> >> On Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken 
> >> wrote:
> >>
> >>> Pierre Joy wrote:
> >>> And what wincache does. It is slower but the request is served.
> >>
> >> WinCache (file cache) if it can't reattach, creates a new shared mem
> >> file
> >
> > I'm not sure how WinCache works, but opcache already implements most
> > necessary functionality.
>
> Sorry for getting to the party late, but let me clarify what WinCache does:
>
> 1. For the opcode cache, if WinCache can't map to the same address, it
> falls back to using process-local (heap) memory for the opcode arrays.
> This is because the elements in the opcode arrays contain absolute
> pointers, and we don't want to spend cycles (a) converting all pointers to
> offsets when copying opcode arrays into shared memory, and then (b)
> converting all offsets back to absolute addresses when copying the opcode
> array back out of shared memory.  WinCache simply gets the pointer to the
> opcode array in shared memory and returns it, unmodified, to the PHP Core
> for execution.
>
> When WinCache falls back to process-local memory, it does NOT create a
> second memory mapped segment, and does not chew additional cross-process
> resources.
>
> 2. WinCache's file cache does not map to an identical address in all
> processes.  We simply map it to *any* address (using NULL for the suggested
> address to MapViewOfFileEx), and then use relative offsets in the cache
> structures for all entries in shared memory.
>
> 3. WinCache's user cache and session cache are similar to the file cache,
> and do not have to map to the same address in all processes. Cache entries
> use relative offsets to locate cache entries, and the values do not contain
> absolute addresses.
>
> Finally, For PHP7, WinCache has removed its opcode cache.  We assume that
> Windows consumers of PHP7 will use Zend Opcache for opcode caching.
>
> Thx!
> --E.
>