>rsync -avnp remote::gif/ `find /home/www/html/ -maxdepth 1 >-name "*.[j,g][pg,if]*"` /tmp/ > >If I run this on the local machine, the rsync server, it takes this >long: > >---> root@server (0.34)# time find /home/www/html/ -maxdepth 1 >-name "*.[j,g][pg,if]*" -type f >/home/www/html/comparestores_2.jpg >/home/www/html/home.jpg >/home/www/html/comparestores_3.jpg >/home/www/html/specialoffer_apparel.jpg >/home/www/html/bike_gary.gif >/home/www/html/gary_bike.gif >/home/www/html/none.gif > >real 0m0.015s >user 0m0.000s >sys 0m0.000s > >However if I run it from a client, it will take forever. Too much to >run, it seems. Our directory structure has well over a million files. >And this is just one directory under /home/www/html. We can't afford the >cpu and system load to traverse everything, this is why I am using the >find command. Shouldn't this work? It does come back with retrieving the >list from the remote server. What OS are you running on both systems?? AFAIK linux with ext2/ext3 has (currently) severe problems with large directories (>5000 files). [Work is done to avoid that: see ext2 directory index patch at http://kernelnewbies.org/~phillips/ ]
Maybe that's your problem. (In my - and strictly my - opinion, a directory with that many files is "unmaintainable". I'd do some partitioning - and if it's only sorting by filetype (.html, .gif, .jpg, ...)) Regards, Phil
