On Wed, May 14, 2014 at 07:48:39AM -0700, Paul Eggert wrote: > Pádraig Brady wrote: > >Also do we always know how much to fallocate? > > Not if we're using compression on the temporaries, no. > > I think a patch along these lines could be worthwhile, if it was simple and > if it actually worked (the current one doesn't).
The current patch only look while files is created, but this is not enough `I agree with you, it must check write(2) and fallback to creating when write(2) will fail with ENOSPC. This is what you mean? > Something along the > following lines, say. When multiple -T options are specified (-T FOO, -T > FOP, -T FOQ, ...) and one of them runs out of disk space when creating a > temporary file FOO/BAR, 'sort' stops creating files in FOO (effectively > removing FOO from the option list) creates a file FOP/BAR instead, and > redoes the process (whatever it was) that sent output to FOO/BAR, sending > the output to FOP/BAR this time. I don't think that redoes is worth it, since when we have ENOSPC it means that we already won't create any more files there, and one file with relatively small size is not a big deal. I also think that dropping direcotory from list is a good idea, user can notice this (using some monitorings) and clean it. For example recently I need to sort relatively huge amount of data, and I don't have enough space for writing all tmp files (why --compress-program not works for me is another story), so I wrote script, that did next: When free space < %3 on some of partitions, I run pkill -STOP sort, and then move some files (that not currently opened by sort(1)) to partition that have more free space, and create symlinks for old locations. When all the free space was eliminated on all available partitions, I archived existed tmp files (that also not opened by sort(1)) and create a pipes, that redirects output of $(gzip -d) into it, and using this hacks sort finished successfully for me. > > I don't have the energy right now to write that, but if someone else wrote > it I'd review it. Thanks for you notes Pádraig. -- Respectfully Azat Khuzhin
