On Thu, Oct 31, 2013 at 2:44 PM, John Napiorkowski <[email protected]>wrote:
> > am calling ->cleanup(1) when we create the HTTP::Body. is that not enough > to cleanup tmp files ? > I haven't look at this in a while, but I think it's described here: https://rt.cpan.org/Public/Bug/Display.html?id=84004 HTTP::Body assumes $self->{upload} exists before deleting, and that might not be created yet. I have my own version for handling 'multipart/form-data' that sets UNLINK => 1. Now, the application/octet-stream handling is another issue. There HTTP::Body uses the default File::Temp (e.g. UNLINK => 1), but I'm still finding a large number of those files left around. In my dev environment I have not been able to make it leave files on /tmp. On production I can run watch 'ls /tmp | wc -l' and see the counts increase and decrease so I know files are being deleted, but every once in a while a file gets left behind. I don't see segfaults in the logs, and I've tested with Apache's MaxRequestPerChild low (so recycling child processes often) and not seeing that leave files behind. I'm going to update our copy of HTTP::Body and put the process ID in the temp file template to essentially namespace and use cron to keep /tmp cleaner. But, I still have yet to figure out why those are left behind. With UNLINK => 1 they should not be left there. File::Temp doesn't appear to check the return value from unlink. They come and go but some stick around: $ for i in $(seq 10); do ls /tmp | wc -l; sleep 2; done 23861 23865 23863 23864 23862 23862 23865 23865 23864 23866 $ ls -lt /tmp | head -2 total 95492 -rw------- 1 tii-rest tii-rest 14 Oct 31 16:40 Nudjp9WDNy $ ls -lt /tmp | tail -2 -rw------- 1 tii-rest tii-rest 16 Oct 28 13:36 NWwxOhwhRW -rw------- 1 tii-rest tii-rest 16 Oct 28 13:35 Ll1Ze0TNPL > > regarding the tmp file thing, wow I have no idea, but I hope you find out > and report it to us! > > Johnn > > > On Friday, October 25, 2013 8:53 AM, Bill Moseley <[email protected]> > wrote: > I have an API where requests can include JSON. HTTP::Body saves those > off to temp files. > > Yesterday got a very large number of errors: > > [ERROR] "Caught exception in engine "Error in tempfile() using > /tmp/XXXXXXXXXX: Have exceeded the maximum number of attempts (1000) to > open temp file/dir > > The File::Temp docs say: > > If you are forking many processes in parallel that are all creating > temporary files, you may need to reset the random number seed using > srand(EXPR) in each child else all the children will attempt to walk > through the same set of random file names and may well cause > themselves to give up if they exceed the number of retry attempts. > > > We are running under mod_perl. Could it be as simple as the procs all > were in sync? I'm just surprised this has not happened before. Is there > another explanation? > > Where would you suggest to call srand()? > > > Another problem, and one I've > commented<https://rt.cpan.org/Public/Bug/Display.html?id=84004>on before, is > that HTTP::Body doesn't use File::Temp's unlink feature and > depends on Catalyst cleaning up. This results in orphaned files left on > temp disk. > > > > > > -- > Bill Moseley > [email protected] > > _______________________________________________ > List: [email protected] > Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst > Searchable archive: > http://www.mail-archive.com/[email protected]/ > Dev site: http://dev.catalyst.perl.org/ > > > > _______________________________________________ > List: [email protected] > Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst > Searchable archive: > http://www.mail-archive.com/[email protected]/ > Dev site: http://dev.catalyst.perl.org/ > > -- Bill Moseley [email protected]
_______________________________________________ List: [email protected] Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/[email protected]/ Dev site: http://dev.catalyst.perl.org/
