good point. I'm running in a VM that is mounting NFS volume. may be part of the problems (slow exec) are related to search path/slow exec. I'll check and revert.
On Sun, Aug 25, 2024, 19:34 Glenn Golden <g...@zplane.com> wrote: > Yair Lenga <yair.le...@gmail.com> [1970-01-01 00:00:00 +0000]: > > > > In my case, I have to bulk-move about 2500 files. This is part of a > > recurring sync job that has to mirror an existing hierarchy into a new > > hierarchy with different naming rules. > > > > It takes no time to create the mapping (even in bash script, case > > statement). When I "pipe" the mapping into "ln" (with xargs) it takes >2 > > min to create the symlinks. Practically, all the time is spent on launch= > > ing "ln". With a custom perl script - it's 3 seconds. > > > > 2c observation: > > Years ago I had a similar weekly need at work, except for an even larger > number of files (10k - 20k or so iirc), and always used a one-liner xargs > script to do the copy. My recollection is that it would complete in "a > few" > seconds (maybe 10s or so). I couldn't find that script, but I just tried > it now manually: Created 2500 randomly named files, each comprising 4kB > random data, and then copied them to new names like this > > $ cat fmap | xargs -L1 cp > > where fmap is the name-mapping file, comprising 2500 lines like > > oldname0 newname0 > oldname1 newname1 > oldname2 newname2 > . . > . . > . . > > It took under 4 seconds, plus another 1-2 seconds for the sync. This was > on > a commodity x86_64 laptop. The target filesystem was the same as original. > Device is a slow 20-year old HDD. > > Since you were reporting 2 min, was wondering what your platform is and > whether there might be something else involved eating the 2 min realtime? > > Glenn >
