Yair Lenga <yair.le...@gmail.com> [1970-01-01 00:00:00 +0000]:
>
> In my case, I have to bulk-move about 2500 files. This is part of a
> recurring sync job that has to mirror an existing hierarchy into a new
> hierarchy with different naming rules.
>
> It takes no time to create the mapping (even in bash script, case
> statement). When I "pipe" the mapping into "ln" (with xargs) it takes >2
> min to create the symlinks. Practically, all the time is spent on launch=
> ing "ln". With a custom perl script - it's 3 seconds.
>
2c observation:
Years ago I had a similar weekly need at work, except for an even larger
number of files (10k - 20k or so iirc), and always used a one-liner xargs
script to do the copy. My recollection is that it would complete in "a few"
seconds (maybe 10s or so). I couldn't find that script, but I just tried
it now manually: Created 2500 randomly named files, each comprising 4kB
random data, and then copied them to new names like this
$ cat fmap | xargs -L1 cp
where fmap is the name-mapping file, comprising 2500 lines like
oldname0 newname0
oldname1 newname1
oldname2 newname2
. .
. .
. .
It took under 4 seconds, plus another 1-2 seconds for the sync. This was on
a commodity x86_64 laptop. The target filesystem was the same as original.
Device is a slow 20-year old HDD.
Since you were reporting 2 min, was wondering what your platform is and
whether there might be something else involved eating the 2 min realtime?
Glenn
