One million files shouldn't be a problem with some planning; we have a
GPFS filesystem with 31 million files that we can backup in 18 hours. We
have some other non-GPFS filesystems with several times that number of
files and we can still get backups done in under 24 hours.
Here's some suggestions:
1. Make sure the TSM server is not a bottleneck. Make sure you have
plenty of CPU, RAM, and that the database is on the fastest disks you
can afford. Make sure the database is spread across multiple volumes
with dedicated disks.
2. If you can, run TSM 6. Running DB2 was a huge win for small file
backup performance for us.
3. Work with the people generating data to split their data up into
multiple directory trees. Add each directory as a virtualmountpoint.
4. Use a network fabric that supports RDMA if you can (i.e. Infiniband,
newer 10GbE). This won't help with metadata lookups, but will make data
transfers much more efficient. You can see whether GPFS is using RDMA by
running "mmfsadm dump verbs".
5. Like with any other backup, work with the users to figure out what
actually needs to be backed up.
-- Skylar Thompson (skyl...@u.washington.edu)
-- Genome Sciences Department, System Administrator
-- Foege Building S046, (206)-685-7354
-- University of Washington School of Medicine
On 02/ 2/12 06:30 AM, Jorge Amil wrote:
Hi everybody,
Does anyone know what is the best way to make a filesystem backup than contains
million files?
Backup image is not posible because is a GPFS filesystem and is not supported.
Thanks in advance
Jorge
