Re: GNU Parallel 20180122 ('Mayon') released [stable]

2018-01-25 Thread Ole Tange
On Tue, Jan 23, 2018 at 4:34 PM, Renan Valieris wrote: > I caught the difference as well, looks like the only files changed were: : >> On Jan 23, 2018 8:51 AM, "Tobias Geerinckx-Rice" wrote: : >> While I was merrily updating the parallel package in GNU Guix, the >> tarball's hash and contents cha

Limiting memory used by parallel?

2018-01-25 Thread hubert depesz lubaczewski
Hi, I'm writing a tool that will make a tarball, and then the tarball is passed to parallel, which splits it into 5GB blocks, and each block is sent to separate pipe. Call looks like: tar cf - /some/directory | parallel -j 5 --pipe --block 5G --recend '' ./handle-single-part.sh "{#}" Where hand

Re: Limiting memory used by parallel?

2018-01-25 Thread hubert depesz lubaczewski
On Thu, Jan 25, 2018 at 11:26:24AM -0500, Joe Sapp wrote: > Hi depesz, > > On Thu, Jan 25, 2018 at 10:33 AM, hubert depesz lubaczewski > wrote: > [snip] > > But it looks that parallel itself is consumming HUGE amount of memory > > - comparable with size of /some/directory itself. > > > > Server t

Re: Limiting memory used by parallel?

2018-01-25 Thread Joe Sapp
Hi depesz, On Thu, Jan 25, 2018 at 10:33 AM, hubert depesz lubaczewski wrote: [snip] > But it looks that parallel itself is consumming HUGE amount of memory > - comparable with size of /some/directory itself. > > Server that I run it on has 64GB of ram, and the script gets killed > after ~ 3 minu

Re: GNU Parallel 20180122 ('Mayon') released [stable]

2018-01-25 Thread Renan Valieris
> Ha! Here I naïvely thought thought that I could update the release > without anyone knowing, since I had not sent out a release email yet. > I never expected anyone would download the file without a release email. actually, I didn't download anything before your email. but it seems the mirror I