2011-07-15 6:21, Daniel Carosone ?????:
um, this is what xargs -P is for ...
Thanks for the hint. True, I don't often use xargs.
However from the man pages, I don't see a "-P" option
on OpenSolaris boxes of different releases, and there
is only a "-p" (prompt) mode. I am not eager to enter
"yes" 400000 times ;)
The way I had this script in practice, I could enter "RM"
once and it worked till the box hung. Even then, a watchdog
script could often have it rebooted without my interaction
so it could continue in the next lifetime ;)
--
Dan.
On Thu, Jul 14, 2011 at 07:24:52PM +0400, Jim Klimov wrote:
2011-07-14 15:48, Frank Van Damme ?????:
It seems counter-intuitive - you'd say: concurrent disk access makes
things only slower - , but it turns out to be true. I'm deleting a
dozen times faster than before. How completely ridiculous. Thank you
:-)
Well, look at it this way: it is not only about singular disk accesses
(i.e. unlike other FSes, you do not in-place modify a directory entry),
with ZFS COW it is about rewriting a tree of block pointers, with any
new writes going into free (unreferenced ATM) disk blocks anyway.
So by hoarding writes you have a chance to reduce mechanical
IOPS required for your tasks. Until you run out of RAM ;)
Just in case it helps, to quickly fire up removals of the specific
directory
after yet another reboot of the box, and not overwhelm it with hundreds
of thousands queued "rm"processes either, I made this script as /bin/RM:
===
#!/bin/sh
SLEEP=10
[ x"$1" != x ]&& SLEEP=$1
A=0
# To rm small files: find ... -size -10
find /export/OLD/PATH/TO/REMOVE -type f | while read LINE; do
du -hs "$LINE"
rm -f "$LINE"&
A=$(($A+1))
[ "$A" -ge 100 ]&& ( date; while [ `ps -ef | grep -wc rm` -gt 50 ]; do
echo "Sleep $SLEEP..."; ps -ef | grep -wc rm ; sleep $SLEEP; ps -ef
| grep -wc rm;
done
date )&& A="`ps -ef | grep -wc rm`"
done ; date
===
Essentially, after firing up 100 "rm attempts" it waits for the "rm"
process count to go below 50, then goes on. Sizing may vary
between systems, phase of the moon and computer's attitude.
Sometimes I had 700 processes stacked and processed quickly.
Sometimes it hung on 50...
HTH,
//Jim
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
