Re: FileSystem and FileContext Janitor, at your service !

2014-03-06 Thread Jay Vyas
I think that is the purpose of the bigtop smoke tests, not the filesystem smoke tests. right? On Thu, Mar 6, 2014 at 12:51 PM, Steve Loughran wrote: > I was thinking to test YARN-hosted apps like MapReduce, we need to see how > they handle filesystems with different consistency/atomicity models,

Re: FileSystem and FileContext Janitor, at your service !

2014-03-06 Thread Steve Loughran
I was thinking to test YARN-hosted apps like MapReduce, we need to see how they handle filesystems with different consistency/atomicity models, and YARN -even MiniYARNCluster -forks things off. If the MR commit logic is isolated, that could be tested in the JUnit JVM. But for other applications -e

Re: FileSystem and FileContext Janitor, at your service !

2014-03-06 Thread Jay Vyas
steve you mentioned: >> but to test YARN it has to be visible across processes. What do you mean by "test yarn"? I think for the FileSystem APIs unit testing, we dont care about YARN, do we? On Thu, Mar 6, 2014 at 6:02 AM, Steve Loughran wrote: > On 5 March 2014 19:07, Jay Vyas wrote: >

Re: FileSystem and FileContext Janitor, at your service !

2014-03-06 Thread Steve Loughran
On 5 March 2014 19:07, Jay Vyas wrote: > Hi HCFS Community :) > > This is Jay... Some of you know me I hack on a broad range of file > system and hadoop ecosystem interoperability stuff. I just wanted to > introduce myself and let you folks know im going to be working to help > clean up the