Pádraig Brady wrote:
> Paolo Bonzini wrote:
>> Maybe we want a --parallel option (too bad -p is taken) for xargs that
>> forces the creation of the number of processes passed with -P or taken
>> from nproc (for example by starting "md5sum $1 $5 $9 ...", "md5sum $2 $6
>> $10 ...", etc.)?
>> That would be an interesting alternative to this core-count proposal...
>
> I'm not sure what you mean here.
> I already suggested to the xargs maintainer that `xargs -P`
> should be equivalent to xargs -P$(nproc).
> `nproc` as an external command would still be useful though.
Here's a patch for that.
It needs to be updated to reference the new gnulib
when Bruno's nproc update hits gnulib.
cheers,
Pádraig.
>From 4089b5c579e3f2c23128cd85b096b5c88759749f Mon Sep 17 00:00:00 2001
From: =?utf-8?q?P=C3=A1draig=20Brady?= <p...@draigbrady.com>
Date: Fri, 30 Oct 2009 10:25:29 +0000
Subject: [PATCH] xargs: auto select the -P argument if not specified
* NEWS: Mention the new functionality.
* doc/find.texi: Likewise. Also move -P beside -n.
* xargs/xargs.1: Likewise.
* import-gnulib.config: Reference the nproc module.
* xargs/xargs.c: Use nproc if no argument passed to -P.
---
NEWS | 5 +++++
doc/find.texi | 15 +++++++++------
import-gnulib.config | 1 +
xargs/xargs.1 | 47 +++++++++++++++++++++++++----------------------
xargs/xargs.c | 12 ++++++++----
5 files changed, 48 insertions(+), 32 deletions(-)
diff --git a/NEWS b/NEWS
index 314022b..cd27a32 100644
--- a/NEWS
+++ b/NEWS
@@ -42,6 +42,11 @@ not be found, respectively.
The -wholename option to find is no longer preferred over -ipath.
+** Functional Enhancements to xargs
+
+One now doesn't need to specify a number to --max-procs (-P) as the
+number of processing units on the system will be used by default.
+
* Major changes in release 4.5.4, 2009-03-10
** Performance changes
diff --git a/doc/find.texi b/doc/find.texi
index 24ee8a8..ca62c14 100644
--- a/doc/find.texi
+++ b/doc/find.texi
@@ -3521,6 +3521,15 @@ Use at most @var{max-args} arguments per command line. Fewer than
option) is exceeded, unless the @samp{-x} option is given, in which
case @code{xargs} will exit.
+...@item --max-pr...@r{[}=@var{max-pro...@r{]}
+...@itemx -...@r{[}@var{max-pro...@r{]}
+Instead of a single process, run up to @var{max-procs} simultaneously.
+If @var{max-procs} is not specified, @code{xargs} will run a process for
+each processing unit detected on the system. If @var{max-procs} is 0,
+...@code{xargs} will run as many processes as possible in parallel. Note you
+probably also need to specify @option{--max-args} or @option{--max-lines}
+so that arguments will be distributed to multiple processes.
+
@item --interactive
@itemx -p
Prompt the user about whether to run each command line and read a line
@@ -3558,12 +3567,6 @@ Print the version number of @code{xargs} and exit.
@itemx -x
Exit if the size (see the @samp{-s} option) is exceeded.
-
-...@item --max-pro...@var{max-procs}
-...@itemx -P @var{max-procs}
-Run simultaneously up to @var{max-procs} processes at once; the default is 1. If
-...@var{max-procs} is 0, @code{xargs} will run as many processes as
-possible simultaneously.
@end table
@node Invoking the shell from xargs
diff --git a/import-gnulib.config b/import-gnulib.config
index 10d85ac..ca77b9e 100644
--- a/import-gnulib.config
+++ b/import-gnulib.config
@@ -58,6 +58,7 @@ mbsstr
mktime
modechange
mountlist
+nproc
pathmax
progname
quotearg
diff --git a/xargs/xargs.1 b/xargs/xargs.1
index 6105f1b..5ba63eb 100644
--- a/xargs/xargs.1
+++ b/xargs/xargs.1
@@ -21,8 +21,8 @@ xargs \- build and execute command lines from standard input
[\fB\-\-max\-args\fR=\fImax-args\fR]
[\fB\-s \fImax-chars\fR]
[\fB\-\-max\-chars\fR=\fImax-chars\fR]
-[\fB\-P \fImax-procs\fR]
-[\fB\-\-max\-procs\fR=\fImax-procs\fR]
+[\fB\-P\fR[\fImax-procs\fR\fR]
+[\fB\-\-max\-procs\fR[=\fImax-procs\fR]]
[\fB\-\-interactive\fR]
[\fB\-\-verbose\fR]
[\fB\-\-exit\fR]
@@ -233,6 +233,29 @@ option is given, in which case
.B xargs will exit.
.TP
.PD 0
+.BR \-\-max\-procs "[=\fImax-procs\fR]"
+.TP
+.PD
+.BI \-P "[max-procs]"
+Instead of a single process, run up to
+.I max-procs
+simultaneously. If
+.I max-procs
+is 0,
+.B xargs
+will run as many processes as
+possible in parallel. If
+.I max-procs
+is not specified, then the number of processing units detected
+on the system will be used. Use the
+.B \-n
+or
+.B \-L
+option with
+.BR \-P ;
+otherwise chances are that only one exec will be done.
+.TP
+.PD 0
.B \-\-interactive
.TP
.PD
@@ -301,26 +324,6 @@ to do anything.
Exit if the size (see the
.B \-s
option) is exceeded.
-.TP
-.PD 0
-.BR \-\-max\-procs "=\fImax-procs\fR"
-.TP
-.PD
-.BI \-P " max-procs"
-Run up to
-.I max-procs
-processes at a time; the default is 1. If
-.I max-procs
-is 0,
-.B xargs
-will run as many processes as
-possible at a time. Use the
-.B \-n
-option or the
-.B \-L
-option with
-.BR \-P ;
-otherwise chances are that only one exec will be done.
.SH "EXAMPLES"
.nf
.B find /tmp \-name core \-type f \-print | xargs /bin/rm \-f
diff --git a/xargs/xargs.c b/xargs/xargs.c
index fb97696..988e81a 100644
--- a/xargs/xargs.c
+++ b/xargs/xargs.c
@@ -95,6 +95,7 @@
#define SIGCHLD SIGCLD
#endif
+#include "nproc.h"
#include "verify.h"
#include "quotearg.h"
#include "findutils-version.h"
@@ -226,7 +227,7 @@ static struct option const longopts[] =
{"verbose", no_argument, NULL, 't'},
{"show-limits", no_argument, NULL, 'S'},
{"exit", no_argument, NULL, 'x'},
- {"max-procs", required_argument, NULL, 'P'},
+ {"max-procs", optional_argument, NULL, 'P'},
{"version", no_argument, NULL, 'v'},
{"help", no_argument, NULL, 'h'},
{NULL, no_argument, NULL, 0}
@@ -478,7 +479,7 @@ main (int argc, char **argv)
bc_use_sensible_arg_max(&bc_ctl);
}
- while ((optc = getopt_long (argc, argv, "+0a:E:e::i::I:l::L:n:prs:txP:d:",
+ while ((optc = getopt_long (argc, argv, "+0a:E:e::i::I:l::L:n:prs:txP::d:",
longopts, (int *) 0)) != -1)
{
switch (optc)
@@ -589,7 +590,10 @@ main (int argc, char **argv)
case 'P':
/* Allow only up to LONG_MAX child processes. */
- proc_max = parse_num (optarg, 'P', 0L, LONG_MAX, 1);
+ if (optarg)
+ proc_max = parse_num (optarg, 'P', 0L, LONG_MAX, 1);
+ else
+ proc_max = num_processors ();
break;
case 'a':
@@ -1442,7 +1446,7 @@ Usage: %s [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]\n\
[-I replace-str] [-i[replace-str]] [--replace[=replace-str]]\n\
[-n max-args] [--max-args=max-args]\n\
[-s max-chars] [--max-chars=max-chars]\n\
- [-P max-procs] [--max-procs=max-procs] [--show-limits]\n\
+ [-P[max-procs]] [--max-procs=max-procs] [--show-limits]\n\
[--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]\n\
[--version] [--help] [command [initial-arguments]]\n"),
program_name);
--
1.6.2.5
