distcp filters and snapshots

2016-12-16 Thread rahul gidwani
I had a quick question for the group. I wanted to backup data to a DR hdfs cluster. With "HDFS-8828 - Utilize Snapshot diff report to build diff copy list in distcp" - this makes it much easier to get the copyFileListing and keep the DR side n sync. I know there are filters which can be applied t

Re: inotify

2016-07-11 Thread rahul gidwani
know how many > bytes the client will write. Would you mind filing a JIRA for this? > > We should also set up inotify events for the snapshot operations at some > point as well. > > regards, > Colin > > On Fri, Jul 1, 2016, at 10:28, rahul gidwani wrote: > > Hello

inotify

2016-07-01 Thread rahul gidwani
Hello kind folks, I was wondering if anyone would be interested in adding a AddBlock Event to the inotify pipeline? We were thinking about using Inotify for hdfs replication to another datacenter. Right now the problem is with the appends. We get a notification when an append starts with an App

INotify and atime

2016-06-08 Thread rahul gidwani
I was testing Inotify locally and noticed that unless I save the FSImage I can't get a MetadataUpdateEvent to get fired. The test is pretty basic: I setup a minicluster with conf.setLong("dfs.access.time.precision", 1); create a file (verify I get a EventType.Create) I read that file and verify