Re: Need to add fs shim to use QFS

2012-10-26 Thread Steve Loughran
On 26 October 2012 01:24, Thilee Subramaniam wrote: > > We have made the changes recommended here, and made available a 'Hadoop > QFS jar' with QFS. This plugin and the QFS libraries will be maintained & > released by the QFS open-source project. > > Please see the download and usage instructions

Re: Need to add fs shim to use QFS

2012-10-25 Thread Thilee Subramaniam
On 10/11/12 8:01 AM, "Steve Loughran" wrote: >On 11 October 2012 00:34, Thilee Subramaniam wrote: > >> >> >> My initial goal was to make Hadoop use QFS the same way it used KFS. >>Since >> Hadoop branch-1 had lib/kfs.xx.jar, I was expecting to include a >> qfs.x.x.jar in the Hadoop release; my f

Re: Need to add fs shim to use QFS

2012-10-11 Thread Steve Loughran
On 11 October 2012 00:34, Thilee Subramaniam wrote: > > > My initial goal was to make Hadoop use QFS the same way it used KFS. Since > Hadoop branch-1 had lib/kfs.xx.jar, I was expecting to include a > qfs.x.x.jar in the Hadoop release; my first patch was to use such jar. But > now I see that Had

Re: Need to add fs shim to use QFS

2012-10-11 Thread Steve Loughran
Good point Steve. This touches on the larger issue of whether it > makes sense to host FS clients for other file systems in Hadoop > itself. I agree with what I think you're getting which is - if we can > handle the testing and integration via external dependencies it would > probably be better

Re: Need to add fs shim to use QFS

2012-10-10 Thread Thilee Subramaniam
On 10/10/12 12:14 PM, "Steve Loughran" wrote: >On 10 October 2012 16:03, Thilee Subramaniam wrote: > >> Hi Steve, >> >> Like Harsh said, HADOOP-8886 addresses removing KFS from apache tree. >> >> But I interpret your suggestion as 'moving qfs.jar out of apache tree, >>and >> keeping the jar in

Re: Need to add fs shim to use QFS

2012-10-10 Thread Eli Collins
On Wed, Oct 10, 2012 at 12:14 PM, Steve Loughran wrote: > On 10 October 2012 16:03, Thilee Subramaniam wrote: > >> Hi Steve, >> >> Like Harsh said, HADOOP-8886 addresses removing KFS from apache tree. >> >> But I interpret your suggestion as 'moving qfs.jar out of apache tree, and >> keeping the

Re: Need to add fs shim to use QFS

2012-10-10 Thread Steve Loughran
On 10 October 2012 16:03, Thilee Subramaniam wrote: > Hi Steve, > > Like Harsh said, HADOOP-8886 addresses removing KFS from apache tree. > > But I interpret your suggestion as 'moving qfs.jar out of apache tree, and > keeping the jar in possibly a maven repo externally. The new fs shim for > QFS

Re: Need to add fs shim to use QFS

2012-10-10 Thread Thilee Subramaniam
Hi Steve, Like Harsh said, HADOOP-8886 addresses removing KFS from apache tree. But I interpret your suggestion as 'moving qfs.jar out of apache tree, and keeping the jar in possibly a maven repo externally. The new fs shim for QFS will pull this jar from the repo upon compilation etc.'. Please c

Re: Need to add fs shim to use QFS

2012-10-10 Thread Harsh J
Hi Steve, Check out https://issues.apache.org/jira/browse/HADOOP-8886 for the KFS removal. Seems relevant to your question here. On Wed, Oct 10, 2012 at 12:52 AM, Steve Loughran wrote: > On 5 October 2012 18:27, Thilee Subramaniam wrote: > >> We at Quantcast have released QFS 1.0 (Quantcast Fil

Re: Need to add fs shim to use QFS

2012-10-09 Thread Steve Loughran
On 5 October 2012 18:27, Thilee Subramaniam wrote: > We at Quantcast have released QFS 1.0 (Quantcast File System) to open > source. This is based on the KFS 0.5 (Kosmos Distributed File System), > a C++ distributed filesystem implementation. KFS plugs into Apache > Hadoop via the 'kfs' shim that

Re: Need to add fs shim to use QFS

2012-10-09 Thread Thilee Subramaniam
Hi Eli, I have attached two patches and submitted them as well. Will the integration engine automatically process 'branch-1' patch or should I do anything spcific for released versions? I set the following for the "Fix versions" field upon patch submission: 1.0.2, 1.0.3, 0.23.3, 2.0.0-alpha, 2.0

Re: Need to add fs shim to use QFS

2012-10-05 Thread Thilee Subramaniam
Thank you Eli. I have just uploaded the patch for branch-1 to JIRA. I'll do the same for trunk soon. If you have any comments or questions on the patch please let me know. Thanks, - Thilee On 10/5/12 10:32 AM, "Eli Collins" wrote: >Hey Thilee, > >Thanks for contributing. We don't process pull

Re: Need to add fs shim to use QFS

2012-10-05 Thread Eli Collins
Hey Thilee, Thanks for contributing. We don't process pull request on the git mirrors, please upload a patch against trunk and branch-1 if you'd like this included in Hadoop 1.x and 2.x releases. More info here: http://wiki.apache.org/hadoop/HowToContribute Thanks, Eli On Fri, Oct 5, 2012 at 10: