Rob Logan wrote:
>
> >> Directory "1" takes between 5-10 minutes for the same command to
> return
> >> (it has about 50,000 files).
>
> > That said, directories with 50K files list quite quickly here.
>
> a directory with 52,705 files lists in half a second here
>
> 36 % time \ls -1 > /dev/n
A user has 5 directories, each has tens of thousands of files, the
largest directory has over a million files. The files themselves are
not very large, here is an "ls -lh" on the directories:
[these are all ZFS-based]
[r...@cluster]# ls -lh
total 341M
drwxr-xr-x+ 2 someone cluster 13K Sep 14 19
We have a SGE array task that we wish to run with elements 1-7.
Each task generates output and takes roughly 20 seconds to 4 minutes
of CPU time. We're doing them on a machine with about 144 8-core nodes,
and we've divvied the job up to do about 500 at a time.
So, we have 500 jobs at a t
