Michael Schuster wrote:
Mike Gerdts wrote:
On Tue, Jan 5, 2010 at 4:34 AM, Mikko Lammi <mikko.la...@lmmz.net> wrote:
Hello,
As a result of one badly designed application running loose for some
time,
we now seem to have over 60 million files in one directory. Good thing
about ZFS is that it allows it without any issues. Unfortunatelly now
that
we need to get rid of them (because they eat 80% of disk space) it seems
to be quite challenging.
Traditional approaches like "find ./ -exec rm {} \;" seem to take
forever
- after running several days, the directory size still says the same.
The
only way how I've been able to remove something has been by giving "rm
-rf" to problematic directory from parent level. Running this command
shows directory size decreasing by 10,000 files/hour, but this would
still
mean close to ten months (over 250 days) to delete everything!
I also tried to use "unlink" command to directory as a root, as a
user who
created the directory, by changing directory's owner to root and so
forth,
but all attempts gave "Not owner" error.
Any commands like "ls -f" or "find" will run for hours (or days) without
actually listing anything from the directory, so I'm beginning to
suspect
that maybe the directory's data structure is somewhat damaged. Is there
some diagnostics that I can run with e.g "zdb" to investigate and
hopefully fix for a single directory within zfs dataset?
In situations like this, ls will be exceptionally slow partially
because it will sort the output.
that's what '-f' was supposed to avoid, I'd guess.
Yes, but unfortunately, the typical reason ls is slow with huge directories
is that it requires a huge amount of memory. Even when not sorting (with
-f), it still allocates a huge amount of memory for each entry listed, and
buffers the output until the directory is entirely read. So typically -f
doesn't help performance much. Improving this would be a great small project
for an OpenSolaris contributor! I filed a couple of bugs for this several
years ago, I can dig them up if anyone is interested.
--matt
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
