Re: [PHP-DEV] readfile() memory usage

2012-05-11 Thread Sebastian Krebs
Hi, 2012/5/11 Simon Schick > Hi, Everyone > > FYI: If you just want to check something before serving a file to the > client, you can also use something called xsendfile. > > Apache: https://tn123.org/mod_xsendfile/ > lighttpd: It's build in :) > nginx: http://wiki.nginx.org/XSendfile It's nam

Re: [PHP-DEV] readfile() memory usage

2012-05-11 Thread Simon Schick
Hi, Everyone FYI: If you just want to check something before serving a file to the client, you can also use something called xsendfile. Apache: https://tn123.org/mod_xsendfile/ lighttpd: It's build in :) nginx: http://wiki.nginx.org/XSendfile Idea: Do what you're doing in your php-script and add

Re: [PHP-DEV] readfile() memory usage

2012-05-04 Thread Paul Reinheimer
Hi Everyone >> So, I think we're back to "urban legend" territory. I've updated the documentation for readfile() to help send more people down the path of checking for output buffering, and disabling that rather than contriving loops with fread(). http://docs.php.net/readfile paul -- Paul

Re: [PHP-DEV] readfile() memory usage

2012-05-04 Thread jpauli
On Tue, May 1, 2012 at 8:44 PM, Paul Reinheimer wrote: > Hi, > >> It sounds like it.  In that case the memory spike is happening in the output >> buffer, where the file is streamed into by readfile() in 8K chunks until the >> output buffer explodes. :-) >> >> So, I think we're back to "urban legen

Re: [PHP-DEV] readfile() memory usage

2012-05-02 Thread Larry Garfield
On 05/01/2012 11:40 AM, Larry Garfield wrote: On 5/1/12 10:01 AM, Paul Reinheimer wrote: Hi All, Unfortunately, you've ignored Uwe's e-mail... The problem is not the PHP version; the problem is that you're buffering unlimited amounts of data. Check your configuration and make sure ob_get_lev

Re: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Paul Reinheimer
Hi, > It sounds like it.  In that case the memory spike is happening in the output > buffer, where the file is streamed into by readfile() in 8K chunks until the > output buffer explodes. :-) > > So, I think we're back to "urban legend" territory. That's good to know. Thanks, and my apologies fo

Re: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Larry Garfield
On 5/1/12 10:01 AM, Paul Reinheimer wrote: Hi All, Unfortunately, you've ignored Uwe's e-mail... The problem is not the PHP version; the problem is that you're buffering unlimited amounts of data. Check your configuration and make sure ob_get_level() returns 0. My apologies in the delay, ob_g

Re: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Paul Reinheimer
Hi All, > Unfortunately, you've ignored Uwe's e-mail... The problem is not the PHP > version; the problem is that you're buffering unlimited amounts of data. > Check your configuration and make sure ob_get_level() returns 0. My apologies in the delay, ob_get_level() returns 1, good catch. phpinfo

Re: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Rasmus Lerdorf
On 05/01/2012 07:27 AM, Rasmus Lerdorf wrote: > On 05/01/2012 06:39 AM, Paul Reinheimer wrote: >> Hi Larry, >> >>> Fascinating. I even verified the md5sum of the file I got on the other end >>> just to be sure. I'll hold off on the blog post then. :-) I look forward >>> to your test setup. >> >>

Re: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Rasmus Lerdorf
On 05/01/2012 06:39 AM, Paul Reinheimer wrote: > Hi Larry, > >> Fascinating. I even verified the md5sum of the file I got on the other end >> just to be sure. I'll hold off on the blog post then. :-) I look forward >> to your test setup. > > The server in question is still on PHP 5.2.13 > > S

Re: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Anthony Ferrara
I know it doesn't really fit this problem in general, but I figured I would point it out. Lighttpd introduced a brilliant concept by letting the server serve that file directly. Basically, instead of using readfile, you would just send a header: X-SendFile: $filename... It's available for Apache

Re: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Gustavo Lopes
On Tue, 01 May 2012 15:39:56 +0200, Paul Reinheimer wrote: Fascinating. I even verified the md5sum of the file I got on the other end just to be sure. I'll hold off on the blog post then. :-) I look forward to your test setup. The server in question is still on PHP 5.2.13 [...] I'll

Re: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Paul Reinheimer
Hi Larry, > Fascinating.  I even verified the md5sum of the file I got on the other end > just to be sure.  I'll hold off on the blog post then. :-)  I look forward > to your test setup. The server in question is still on PHP 5.2.13 Script: http://www.php.net/unsub.php

RE: [PHP-DEV] readfile() memory usage

2012-05-01 Thread Uwe Schindler
> Hi Larry, > > > 4) So given #2 and #3, the "readfile() will kill your memory, don't use it" > > line is a persistent urban legend that belongs on Snopes as debunked. > > Looping on fread() for performance is a red herring. > > I implemented this earlier this very year to avoid memory issues (a

Re: [PHP-DEV] readfile() memory usage

2012-04-30 Thread Larry Garfield
On 04/30/2012 07:37 PM, Paul Reinheimer wrote: Hi Larry, 4) So given #2 and #3, the "readfile() will kill your memory, don't use it" line is a persistent urban legend that belongs on Snopes as debunked. Looping on fread() for performance is a red herring. I implemented this earlier this very

Re: [PHP-DEV] readfile() memory usage

2012-04-30 Thread Paul Reinheimer
Hi Larry, > 4) So given #2 and #3, the "readfile() will kill your memory, don't use it" > line is a persistent urban legend that belongs on Snopes as debunked. >  Looping on fread() for performance is a red herring. I implemented this earlier this very year to avoid memory issues (a quick look at

RE: [PHP-DEV] readfile() memory usage

2012-04-30 Thread Uwe Schindler
th work. - Uwe Schindler theta...@php.net - http://www.php.net NSAPI SAPI developer Bremen, Germany > -Original Message- > From: Larry Garfield [mailto:la...@garfieldtech.com] > Sent: Monday, April 30, 2012 7:16 PM > To: internals@lists.php.net > Subject: Re: [PHP-DEV] readfi

Re: [PHP-DEV] readfile() memory usage

2012-04-30 Thread Larry Garfield
Thanks all. So it sounds like the answer is: 1) readfile() has always been memory-safe as far as PHP is concerned. 2) Because it uses mmap(), GFL trying to understand its memory usage from top. 3) Operating systems have gotten better at such things in the past decade. 4) So given #2 and #3, th

Re: [PHP-DEV] readfile() memory usage

2012-04-30 Thread Sherif Ramadan
> Mmapping of course uses memory, but the memory used here is not from PHP's > memory manager, it's memory that's already used for the O/S cache. The > memory mapping used here does not enforce loading the file contents into O/S > cache; it just gets a virtual address into the O/S cache. If the act

RE: [PHP-DEV] readfile() memory usage

2012-04-30 Thread Uwe Schindler
Hi, > mmap may use "normal" memory too, depending on the options (not sure > which are used exactly with readfile or stream's mmap). Mmapping of course uses memory, but the memory used here is not from PHP's memory manager, it's memory that's already used for the O/S cache. The memory mapping use

Re: [PHP-DEV] readfile() memory usage

2012-04-30 Thread Pierre Joye
hi! On Mon, Apr 30, 2012 at 8:25 AM, Uwe Schindler wrote: > - If the underlying stream allows MMAP, it will use memory mapping (mapping > file to *virtual* memory) and copy the mapped buffer to output. Please note > memory mapping does *not* load the file into memory, it only *maps* the file > c

Re: [PHP-DEV] readfile() memory usage

2012-04-29 Thread Sherif Ramadan
> So, I've been reading articles for a decade now that say that readfile() is > great and wonderful except for memory usage.  Specifically, that it reads a > file into memory entirely, and then prints it to stdout from there.  So if > you're outputing a big file you will hit your memory limit and k

RE: [PHP-DEV] readfile() memory usage

2012-04-29 Thread Uwe Schindler
Hi, Readfile() is internally implemented in the same way like fpassthru() (actually the same backend function is called, readfile only opening a stream before delegating to passthru). Both methods delegate from PHP user-space to an internal streams API methods php_stream_passthru(). This one has 2