On Wed, Oct 10, 2012 at 12:14 PM, Steve Loughran <ste...@hortonworks.com> wrote: > On 10 October 2012 16:03, Thilee Subramaniam <thi...@quantcast.com> wrote: > >> Hi Steve, >> >> Like Harsh said, HADOOP-8886 addresses removing KFS from apache tree. >> >> But I interpret your suggestion as 'moving qfs.jar out of apache tree, and >> keeping the jar in possibly a maven repo externally. The new fs shim for >> QFS will pull this jar from the repo upon compilation etc.'. >> > > I think the main question is if its external, what actually needs to be > done w.r.t. Hadoop's build itself. Is your goal to include the shim JAR in > the normal Hadoop releases? As that's what I'm not sure is needed -no other > non-HDFS filesystem needs that to work with Hadoop. What matters more is to > hook a local Jenkins server to do the nightly builds and tests of hadoop > branch-1 and trunk onto your filesystem, test out all the releases, and > then file bug reports if there have been any regressions.
Good point Steve. This touches on the larger issue of whether it makes sense to host FS clients for other file systems in Hadoop itself. I agree with what I think you're getting which is - if we can handle the testing and integration via external dependencies it would probably be better to have the Hadoop client code live and ship as part of the other projects since it's more likely to be maintained there. Perhaps start a DISCUSS thread on common-dev since this pertains to other file systems aside from QFS? Thanks, Eli
