Hello Rick, I cannot give you any special insights into Hadoop backup, but for your excludes, consider using exclude.dir instead. This will prevent the client from traversing into those directories at all. Benefits include: The subdirectory structure is also excluded from backup, and backup run time is improved (the significance of the amount of improvement will depend on how deep those directory structures are and how many files are in them):
exclude.dir /cld1/impala exclude.dir /cld2/impala ... exclude.dir /cld12/impala and the same for the "yarn" directories. Best regards, Andy ____________________________________________________________________________ Andrew Raibeck | IBM Spectrum Protect Level 3 | stor...@us.ibm.com "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 2018-08-28 10:51:05: > From: "Rhodes, Richard L." <rrho...@firstenergycorp.com> > To: ADSM-L@VM.MARIST.EDU > Date: 2018-08-28 10:52 > Subject: hadoop cluster > Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> > > Hello! > > Someone purchased a 12 node Hadoop cluster. The BA client was > installed on each node and it's flooding TSM with TB of backups per > night. I don't know who's Hadoop implementation we have, but it > runs on RH Linux. > > The unix admin is working with the users of the cluster on excluding > unneeded files. Currently he is going to add these excludes to each node: > exclude /cld*/impala/.../* > exclude /cld*/yarn/.../* > (the dirs on each node are cld1 thru cld12) > > Any thoughts/comments/experiences/recommendations on backing up a > Hadoop cluster are appreciated! > > > Thanks! > > rick > ------------------------------------------------------------------------------ > > The information contained in this message is intended only for the > personal and confidential use of the recipient(s) named above. If > the reader of this message is not the intended recipient or an agent > responsible for delivering it to the intended recipient, you are > hereby notified that you have received this document in error and > that any review, dissemination, distribution, or copying of this > message is strictly prohibited. If you have received this > communication in error, please notify us immediately, and delete the > original message. >
