I'm fine with the concept; I haven't looked too hard at the
implementation, but it looks like it's doing the right thing.
--Wez.
On 9/7/05, Andi Gutmans <[EMAIL PROTECTED]> wrote:
> Hey all,
>
> Has anyone had a chance to review this patch.
> I think it'd be good to apply it. I've seen quite a few instances
> where PHP processes blow up during a request and then hand around
> with 100MB+. No one here had that problem?
>
> Andi
>
> At 09:27 AM 8/30/2005, Alexander Veremyev wrote:
> >Some people have mentioned having problems where idle PHP processes
> >hang around without freeing their memory (in a FastCGI or Apache module
> >modes).
> >
> >As I have found, it's not really a memory fragmentation problem.
> >The core of the problem is the fact, that free() doesn't return
> >memory to the OS immediately. So, if we have memory hungry PHP
> >scripts and use PHP in a FastCGI or Apache module environment, then
> >large amount of memory could still remain allocated for a process
> >while it's waiting for the next HTTP request.
> >
> >Attached is a patch which resolves this problem by forcing the
> >memory allocator to return free pages to the operating system at the
> >end of the request. Especially in instances where some of your PHP
> >scripts use a lot of memory, this maybe be quite useful in reducing
> >the machine's overall memory consumption at a given time.
> >
> >I think it's probably best for this explicit memory collector mode
> >not to be used in the multithreaded environment (for performance
> >reasons), but the patch works properly if ZTS is enabled.
> >
> >Two new configuration variables could be set in a php.ini:
> >
> >memory_trim_limit (default is 1M) - is a minimum memory grow per
> >request limit until malloc_trim mechanism will be invoked. "0" value
> >disables it.
> >
> >memory_grow_limit (default is 128M) - is a memory grow limit. If
> >process reaches it, then force_exit SAPI global is turned on and
> >could be used in SAPI modules to force process exit. This is for the
> >case when we have persistent objects on the top of memory or memory
> >leaks in external libraries. "0" value disables it.
> >
> >Patch also includes processing of force_exit SAPI global for CGI
> >SAPI module (for FastCGI mode), Apache SAPI module and Apache 2.0
> >Handler SAPI module (for prefork Apache2 mode).
> >
> >Please review it, try it and send me feedback.
> >
> >With best regards,
> > Alexander Veremyev.
> >
> >
> >
> >Index: php-src/Zend/zend_alloc.c
> >===================================================================
> >RCS file: /repository/ZendEngine2/zend_alloc.c,v
> >retrieving revision 1.144
> >diff -u -r1.144 zend_alloc.c
> >--- php-src/Zend/zend_alloc.c 3 Aug 2005 13:30:47 -0000 1.144
> >+++ php-src/Zend/zend_alloc.c 9 Aug 2005 23:18:58 -0000
> >@@ -23,6 +23,7 @@
> > #include "zend_alloc.h"
> > #include "zend_globals.h"
> > #include "zend_fast_cache.h"
> >+#include "SAPI.h"
> > #ifdef HAVE_SIGNAL_H
> > # include <signal.h>
> > #endif
> >@@ -455,6 +456,52 @@
> > }
> >
> >
> >+ZEND_API int zend_set_memory_trim_limit(unsigned int memory_trim_limit)
> >+{
> >+ TSRMLS_FETCH();
> >+
> >+ AG(memory_trim_limit) = memory_trim_limit;
> >+ return SUCCESS;
> >+}
> >+
> >+
> >+ZEND_API int zend_set_memory_grow_limit(unsigned int memory_grow_limit)
> >+{
> >+ TSRMLS_FETCH();
> >+
> >+ AG(memory_grow_limit) = memory_grow_limit;
> >+ return SUCCESS;
> >+}
> >+
> >+
> >+#if defined(ZEND_WIN32)
> >+static inline unsigned int HeapCommittedSize( HANDLE memory_heap )
> >+{
> >+ PROCESS_HEAP_ENTRY heap_entry;
> >+ unsigned int mem_size;
> >+ int cnt;
> >+
> >+ cnt=0;
> >+
> >+ mem_size = 0;
> >+ memset(&heap_entry,0,sizeof(heap_entry));
> >+ heap_entry.wFlags = PROCESS_HEAP_REGION;
> >+
> >+ while( HeapWalk(memory_heap,&heap_entry) ) {
> >+ if( heap_entry.wFlags & PROCESS_HEAP_REGION ) {
> >+ mem_size += heap_entry.Region.dwCommittedSize;
> >+ }
> >+ }
> >+
> >+ if( GetLastError() != ERROR_NO_MORE_ITEMS ) {
> >+ fprintf(stderr, "HeapCommittedSize: Heap is corrupted\n" );
> >+ return 0;
> >+ }
> >+ return mem_size;
> >+}
> >+#endif
> >+
> >+
> > ZEND_API void start_memory_manager(TSRMLS_D)
> > {
> > AG(head) = NULL;
> >@@ -483,6 +530,15 @@
> > memset(AG(cache_stats), 0, sizeof(AG(cache_stats)));
> > memset(AG(fast_cache_stats), 0, sizeof(AG(fast_cache_stats)));
> > #endif
> >+
> >+#ifdef ZEND_WIN32
> >+ /* Heap is just created, so it costs us nothing to get heap size */
> >+ AG(startup_memory) = AG(request_start_memory) =
> >HeapCommittedSize( AG(memory_heap) );
> >+#elif HAVE_SBRK
> >+ AG(startup_memory) = AG(request_start_memory) = (unsigned
> >int)sbrk(0);
> >+#else
> >+ AG(startup_memory) = AG(request_start_memory) = 0;
> >+#endif
> > }
> >
> >
> >@@ -648,6 +704,39 @@
> > return;
> > }
> > #endif
> >+
> >+ if ( !full_shutdown && (AG(memory_trim_limit) !=
> >0 || AG(memory_grow_limit) != 0) ) {
> >+ unsigned int process_memory;
> >+#ifdef ZEND_WIN32
> >+ process_memory = HeapCommittedSize( AG(memory_heap) );
> >+ if( AG(memory_trim_limit) &&
> >+ (int)(process_memory -
> >AG(request_start_memory)) > AG(memory_trim_limit) ) {
> >+ HeapCompact( AG(memory_heap), HEAP_NO_SERIALIZE );
> >+ process_memory = HeapCommittedSize( AG(memory_heap)
> >);
> >+ }
> >+#elif HAVE_SBRK
> >+ process_memory = (unsigned int)sbrk(0);
> >+ #if HAVE_MALLOC_TRIM
> >+ if( AG(memory_trim_limit) &&
> >+ (int)(process_memory -
> >AG(request_start_memory)) > AG(memory_trim_limit) ) {
> >+ malloc_trim(0);
> >+ process_memory = (unsigned int)sbrk(0);
> >+ }
> >+ #endif
> >+#else
> >+ process_memory = 0;
> >+#endif
> >+
> >+ if( AG(memory_grow_limit) &&
> >+ (int)(process_memory - AG(startup_memory)) >
> >AG(memory_grow_limit) ) {
> >+ SG(force_exit) = 1;
> >+ }
> >+
> >+ /* Current process memory size should be used as
> >request_start_memory
> >+ * for next request
> >+ */
> >+ AG(request_start_memory) = process_memory;
> >+ }
> > }
> >
> >
> >Index: php-src/Zend/zend_alloc.h
> >===================================================================
> >RCS file: /repository/ZendEngine2/zend_alloc.h,v
> >retrieving revision 1.63
> >diff -u -r1.63 zend_alloc.h
> >--- php-src/Zend/zend_alloc.h 3 Aug 2005 13:30:47 -0000 1.63
> >+++ php-src/Zend/zend_alloc.h 9 Aug 2005 23:18:58 -0000
> >@@ -172,6 +172,9 @@
> >
> > ZEND_API int zend_set_memory_limit(unsigned int memory_limit);
> >
> >+ZEND_API int zend_set_memory_trim_limit(unsigned int memory_trim_limit);
> >+ZEND_API int zend_set_memory_grow_limit(unsigned int memory_grow_limit);
> >+
> > ZEND_API void start_memory_manager(TSRMLS_D);
> > ZEND_API void shutdown_memory_manager(int silent, int
> > full_shutdown TSRMLS_DC);
> >
> >Index: php-src/Zend/zend_globals.h
> >===================================================================
> >RCS file: /repository/ZendEngine2/zend_globals.h,v
> >retrieving revision 1.141
> >diff -u -r1.141 zend_globals.h
> >--- php-src/Zend/zend_globals.h 3 Aug 2005 13:30:52 -0000 1.141
> >+++ php-src/Zend/zend_globals.h 9 Aug 2005 23:18:58 -0000
> >@@ -261,6 +261,10 @@
> > #ifdef ZEND_MM
> > zend_mm_heap mm_heap;
> > #endif
> >+ unsigned int memory_trim_limit;
> >+ unsigned int memory_grow_limit;
> >+ unsigned int startup_memory;
> >+ unsigned int request_start_memory;
> > };
> >
> > struct _zend_scanner_globals {
> >Index: php-src/Zend/Zend.m4
> >===================================================================
> >RCS file: /repository/ZendEngine2/Zend.m4,v
> >retrieving revision 1.58
> >diff -u -r1.58 Zend.m4
> >--- php-src/Zend/Zend.m4 14 Jun 2005 12:23:26 -0000 1.58
> >+++ php-src/Zend/Zend.m4 9 Aug 2005 23:18:58 -0000
> >@@ -313,3 +313,6 @@
> >
> > ])
> >
> >+
> >+AC_CHECK_FUNC(sbrk,[AC_DEFINE(HAVE_SBRK, 1,[ ])])
> >+AC_CHECK_FUNC(malloc_trim,[AC_DEFINE(HAVE_MALLOC_TRIM, 1,[ ])])
> >Index: php-src/main/SAPI.h
> >===================================================================
> >RCS file: /repository/php-src/main/SAPI.h,v
> >retrieving revision 1.114
> >diff -u -r1.114 SAPI.h
> >--- php-src/main/SAPI.h 3 Aug 2005 14:08:28 -0000 1.114
> >+++ php-src/main/SAPI.h 9 Aug 2005 23:18:59 -0000
> >@@ -129,6 +129,7 @@
> > zend_bool sapi_started;
> > time_t global_request_time;
> > HashTable known_post_content_types;
> >+ int force_exit;
> > } sapi_globals_struct;
> >
> >
> >Index: php-src/main/php_globals.h
> >===================================================================
> >RCS file: /repository/php-src/main/php_globals.h,v
> >retrieving revision 1.98
> >diff -u -r1.98 php_globals.h
> >--- php-src/main/php_globals.h 3 Aug 2005 14:08:33 -0000 1.98
> >+++ php-src/main/php_globals.h 9 Aug 2005 23:18:59 -0000
> >@@ -76,6 +76,9 @@
> > long memory_limit;
> > long max_input_time;
> >
> >+ long memory_trim_limit;
> >+ long memory_grow_limit;
> >+
> > zend_bool track_errors;
> > zend_bool display_errors;
> > zend_bool display_startup_errors;
> >Index: php-src/main/main.c
> >===================================================================
> >RCS file: /repository/php-src/main/main.c,v
> >retrieving revision 1.639
> >diff -u -r1.639 main.c
> >--- php-src/main/main.c 3 Aug 2005 14:08:30 -0000 1.639
> >+++ php-src/main/main.c 9 Aug 2005 23:18:59 -0000
> >@@ -119,6 +119,31 @@
> > /* }}} */
> > #endif
> >
> >+/* {{{ PHP_INI_MH
> >+ */
> >+static PHP_INI_MH(OnChangeMemoryTrimLimit)
> >+{
> >+ if (new_value) {
> >+ PG(memory_trim_limit) = zend_atoi(new_value,
> >new_value_length);
> >+ } else {
> >+ PG(memory_trim_limit) = 1024*1024;
> >+ }
> >+ return zend_set_memory_trim_limit(PG(memory_trim_limit));
> >+}
> >+/* }}} */
> >+
> >+/* {{{ PHP_INI_MH
> >+ */
> >+static PHP_INI_MH(OnChangeMemoryGrowLimit)
> >+{
> >+ if (new_value) {
> >+ PG(memory_grow_limit) = zend_atoi(new_value,
> >new_value_length);
> >+ } else {
> >+ PG(memory_grow_limit) = 128*1024*1024;
> >+ }
> >+ return zend_set_memory_grow_limit(PG(memory_grow_limit));
> >+}
> >+/* }}} */
> >
> > /* {{{ php_disable_functions
> > */
> >@@ -313,6 +338,8 @@
> > #if MEMORY_LIMIT
> > PHP_INI_ENTRY("memory_limit",
> > "8M", PHP_INI_ALL, OnChangeMemoryLimit)
> > #endif
> >+ PHP_INI_ENTRY("memory_trim_limit",
> >"1M", PHP_INI_ALL, OnChangeMemoryTrimLimit)
> >+ PHP_INI_ENTRY("memory_grow_limit",
> >"128M", PHP_INI_ALL, OnChangeMemoryGrowLimit)
> > PHP_INI_ENTRY("precision",
> > "14", PHP_INI_ALL, OnSetPrecision)
> > PHP_INI_ENTRY("sendmail_from",
> > NULL, PHP_INI_ALL, NULL)
> > PHP_INI_ENTRY("sendmail_path", DEFAULT_SENDMAIL_PATH,
> > PHP_INI_SYSTEM, NULL)
> >Index: php-src/sapi/cgi/cgi_main.c
> >===================================================================
> >RCS file: /repository/php-src/sapi/cgi/cgi_main.c,v
> >retrieving revision 1.267
> >diff -u -r1.267 cgi_main.c
> >--- php-src/sapi/cgi/cgi_main.c 3 Aug 2005 11:12:17 -0000 1.267
> >+++ php-src/sapi/cgi/cgi_main.c 9 Aug 2005 23:18:59 -0000
> >@@ -1644,7 +1644,7 @@
> > if (!fastcgi) break;
> > /* only fastcgi will get here */
> > requests++;
> >- if (max_requests && (requests == max_requests)) {
> >+ if ( SG(force_exit) || (max_requests &&
> >(requests == max_requests)) ) {
> > FCGX_Finish_r(&request);
> > #ifndef PHP_WIN32
> > if (bindpath) {
> >Index: php-src/sapi/apache/sapi_apache.c
> >===================================================================
> >RCS file: /repository/php-src/sapi/apache/sapi_apache.c,v
> >retrieving revision 1.47
> >diff -u -r1.47 sapi_apache.c
> >--- php-src/sapi/apache/sapi_apache.c 3 Aug 2005 14:08:46 -0000 1.47
> >+++ php-src/sapi/apache/sapi_apache.c 9 Aug 2005 23:19:00 -0000
> >@@ -58,6 +58,10 @@
> > zend_try {
> > php_request_shutdown(NULL);
> > } zend_end_try();
> >+
> >+ if( SG(force_exit) ) {
> >+ ap_child_terminate( ((request_rec *)SG(server_context)) );
> >+ }
> >
> > return retval;
> > }
> >Index: php-src/sapi/apache2handler/sapi_apache2.c
> >===================================================================
> >RCS file: /repository/php-src/sapi/apache2handler/sapi_apache2.c,v
> >retrieving revision 1.57
> >diff -u -r1.57 sapi_apache2.c
> >--- php-src/sapi/apache2handler/sapi_apache2.c 3 Aug 2005 14:08:48
> >-0000 1.57
> >+++ php-src/sapi/apache2handler/sapi_apache2.c 9 Aug 2005 23:19:00 -0000
> >@@ -592,6 +592,12 @@
> > ctx->r = parent_req;
> > }
> >
> >+#ifndef ZTS
> >+ if (SG(force_exit)) {
> >+ exit(0);
> >+ }
> >+#endif
> >+
> > return OK;
> > }
> >
> >
> >
> >
> >
> >--
> >PHP Internals - PHP Runtime Development Mailing List
> >To unsubscribe, visit: http://www.php.net/unsub.php
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
