GNU Parallel - Error:Argument list too long

2017-01-16 Thread Samdani A
Dear all, I used GNU parallel to process 144766 files using 8 jobs run in parallel. But the following error arising on starting the process. parallel -j 8 "do something" ::: *.txt bash: /usr/local/bin/parallel: Argument list too long Can anyone tell the maximum input files which can be given for

Re: GNU Parallel - Error:Argument list too long

2017-01-16 Thread Shlomi Fish
Hi Samdani A., On Mon, 16 Jan 2017 16:57:47 +0530 Samdani A wrote: > Dear all, > > I used GNU parallel to process 144766 files using 8 jobs run in parallel. > But the following error arising on starting the process. > > parallel -j 8 "do something" ::: *.txt > bash: /usr/local/bin/parallel: Ar

Re: GNU Parallel - Error:Argument list too long

2017-01-16 Thread Samdani A
Hi shlomi Thank you for the help and clarifying the doubt. The directory only contains txt files. Can i skip th grep step proceed further like this e.g; ls | [ parallel here ] On Mon, Jan 16, 2017 at 5:19 PM, Shlomi Fish wrote: > Hi Samdani A., > > On Mon, 16 Jan 2017 16:57:47 +0530 > Samdani

Re: GNU Parallel - Error:Argument list too long

2017-01-16 Thread Shlomi Fish
Hi Samdani, On Mon, 16 Jan 2017 17:45:55 +0530 Samdani A wrote: > Hi shlomi > > Thank you for the help and clarifying the doubt. > > The directory only contains txt files. Can i skip th grep step proceed > further like this e.g; > ls | [ parallel here ] > I believe so. Don't see why not if t

Re: GNU Parallel - Error:Argument list too long

2017-01-16 Thread Samdani A
Hi shlomi, Thank you. On Mon, Jan 16, 2017 at 10:34 PM, Shlomi Fish wrote: > Hi Samdani, > > On Mon, 16 Jan 2017 17:45:55 +0530 > Samdani A wrote: > > > Hi shlomi > > > > Thank you for the help and clarifying the doubt. > > > > The directory only contains txt files. Can i skip th grep step p