Hi Matt, Thanks for looking into the problem, but I afraid that the proposed solution won't solve the problem completely, but may introduce new problems. At first, we might to reserve N times more memory. Then different SHM segments might contain different data. opcache_reset(), opcache_invalidate() and others will affect only one segment.
I think the perfect solution would require converting all stored data in position independent format, but this hardly ever is going to be done even for php-8. We may fallback to use file-cache introduced in php-7. (this should work out of the box or with minimal changes). Also. if we can't map SHM into desired address space, we may map it in any other address and copy data into the process memory similar to file-cache. Finally we may try to use /DYNAMICBASE:NO Thanks. Dmitry. On Tue, Sep 29, 2015 at 2:55 AM, Matt Ficken <themattfic...@gmail.com> wrote: > Great. The locking issue is fixed. > > But, it sometimes still fails to reattach to the base address. Because of > ASLR, extra DLLs, etc... the existing base address may not be usable in > other processes that need to reattach. When I see this issue I'm creating a > series of processes (for test runs) and when a process fails to reattach, > it happens to all the following processes for a few minutes. ASLR waits a > few minutes before changing the randomized memory layout when creating a > new process. When this happens to production web sites, it often sounds > like this is the case, as the web site becomes unusable for a while. > > Exiting when the base address can't be reattached to, is not a good way to > handle this case, even in a FastCGI scenario. > > The contract for create_segments() is to reattach successfully or return > ALLOC_FAILED. > > > Option 1: change create_segments() contract and > zend_shared_alloc_startup() so that create_segments() can `disable` > OpCache, possibly setting zend_accel_globals.enable=false and adding > additional checks. This eliminates the benefit of using OpCache, likely for > several processes, causing a major performance drop for a web site. > > Option 2: creating a backup, 2nd backup, etc... OpCache in this case. > There may be up to 6 OpCaches(5 backups) with 4 attempts each. This fix is > simpler.This allows processes to share a backup OpCache, a web site will > slow down only to compile the scripts again(at the expense of some > additional memory). > > There have been some feature requests to be able to create separate > OpCaches anyway. Shared web hosters would like to have different OpCaches > for different sites, but this would require a new INI directive which is > too late to add for 7, and for machines that run different PHP builds. > > This patch passes my functional, stress and performance tests. > > I've added it to the PR 1536: > https://github.com/php/php-src/commit/7cbebc747e5c5f824a6e0336cc044ce4dc4c8f9f > > > Regards > -M > > > On Thu, Sep 24, 2015 at 1:29 AM, Dmitry Stogov <dmi...@zend.com> wrote: > >> Ops, I forgot to revert the exit condition. >> >> >> http://git.php.net/?p=php-src.git;a=commitdiff;h=2d55e8c186ef1034c2af64478da8f23dbeb28be9 >> >> >> >> On Thu, Sep 24, 2015 at 11:00 AM, Dmitry Stogov <dmi...@zend.com> wrote: >> >>> hi Matt, >>> >>> Thanks. >>> I also moved the exit condition to be before Sleep(). >>> This should prevent race condition on last iteration (after unlocked >>> sleep process should try to reattach). >>> Committed to PHP-7.0 and master. >>> >>> >>> http://git.php.net/?p=php-src.git;a=commitdiff;h=262160e0e9919dce914df2f0643c4b16ca137454 >>> >>> If this really works, we should backport this into php-5.* as well. >>> >>> Thanks. Dmitry. >>> >>> On Thu, Sep 24, 2015 at 8:34 AM, Matt Ficken <themattfic...@gmail.com> >>> wrote: >>> >>>> Ok, I have a new PR just to unlock around Sleep(): >>>> https://github.com/php/php-src/pull/1536 >>>> >>>> Just by unlocking around Sleep() the OpenFileMapping loop will work. >>>> Holding the lock in that loop will not only block detach_segments() in >>>> other PHP processes but also reattaching in accel_startup(). >>>> >>>> Checking the base address used when this issue occurs during some of my >>>> stress testing, most of the time the base address used is the first in >>>> vista_mapping_base_set. Its typically the timing between the locking and >>>> mapped file operations between processes more often than the base address >>>> selection. >>>> >>>> Especially ASLR, also PECL DLLs, and other factors will still cause >>>> this issue occasionally, but this fixes a majority of my occurrences. >>>> >>>> Regards >>>> -M >>>> >>>> >>>> On Wed, Sep 23, 2015 at 2:04 AM, Dmitry Stogov <dmi...@zend.com> wrote: >>>> >>>>> Hi Matt, >>>>> >>>>> It looks like with your patch, the same file may be mapped to >>>>> different virtual addresses of different processes and this won't work. >>>>> I think usage of vista_mapping_base_set[] in your patch is wrong. >>>>> >>>>> Do I understand properly, that zend_shared_alloc_unlock/lock_win32() >>>>> around Sleep() is the main part of the patch? >>>>> >>>>> Please, next time, try to make PR diffs as small as possible (without >>>>> messing real fix with formatting and rearranging changes). >>>>> >>>>> Thanks. Dmitry. >>>>> >>>>> >>>>> >>>>> On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken <themattfic...@gmail.com >>>>> > wrote: >>>>> >>>>>> I want to increase visibility for my PR 1531, >>>>>> https://github.com/php/php-src/pull/1531, my patch for fixing an >>>>>> intermittent OpCache issue on Windows. >>>>>> >>>>>> Details, etc... are on the PR. >>>>>> >>>>>> Regards >>>>>> -M >>>>>> >>>>> >>>>> >>>> >>> >> >
