Hi Andy, When I ran Hadoop on Ceph, I was able to complete a run of Terasort. We didn't run HBase.
After I submitted a patch to the Ceph mailing list, Noah Watkins (CC'd) took over developing the file system class in Hadoop's code base. I don't know what's become of the work since then. The JIRA is here: https://issues.apache.org/jira/browse/HADOOP-6779 The last I heard, which was a while ago, there was a snag with the URI scheme (ceph:// vs. file:// or something similar) which had a subtle but challenging complexity. Noah is more able to answer questions at this point. He has posted Java RADOS bindings to the Ceph mailing list recently, but I'm not sure what he has been up to aside from that. --Alex On Aug 27, 2011, at 02:07 , Andrew Purtell wrote: > Hi Alex, > >> UC Santa Cruz contributed a ;login: article describing replacing HDFS with >> Ceph. (I was one of the authors.) One of the key architectural advantages >> of >> Ceph over HDFS is that Ceph distributes its metadata service over multiple >> metadata servers. > > Were you able to maintain stable operation with Ceph at a high load? Did you > try running HBase? > > Best regards, > > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein (via > Tom White) > > > ----- Original Message ----- >> From: Alex Nelson <ajnel...@cs.ucsc.edu> >> To: George Kousiouris <gkous...@mail.ntua.gr> >> Cc: common-dev@hadoop.apache.org; common-u...@hadoop.apache.org; Carlos >> Maltzahn <carl...@soe.ucsc.edu> >> Sent: Friday, August 26, 2011 2:44 AM >> Subject: Re: Any related paper on how to resolve hadoop SPOF issue? >> >> Hi George, >> >> UC Santa Cruz contributed a ;login: article describing replacing HDFS with >> Ceph. (I was one of the authors.) One of the key architectural advantages >> of >> Ceph over HDFS is that Ceph distributes its metadata service over multiple >> metadata servers. >> >> I hope that helps. >> >> --Alex >> >> >> On Aug 25, 2011, at 02:54 , George Kousiouris wrote: >> >>> >>> Hi, >>> >>> many thanks!! >>> >>> I also found this one, which has some related work also with other efforts, >> in case it is helpful for someone else: >>> >> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1 >>> >>> >>> BR, >>> George >>> >>> On 8/25/2011 12:46 PM, Nan Zhu wrote: >>>> Hope it helps >>>> >>>> http://www.springerlink.com/content/h17r882710314147/ >>>> >>>> Best, >>>> >>>> Nan >>>> >>>> On Thu, Aug 25, 2011 at 5:43 PM, George >> Kousiouris<gkous...@mail.ntua.gr>wrote: >>>> >>>>> Hi guys, >>>>> >>>>> We are currently in the process of writing a paper regarding hadoop >> and we >>>>> would like to reference any attempt to remove the single point of >> failure of >>>>> the Namenode. We have found in various presentations some efforts >> (like >>>>> dividing the namespace between more than one namenodes if i >> remember >>>>> correctly) but the search for a concrete paper on the issue came to >> nothing. >>>>> >>>>> Is anyone aware or has participated in such an effort? >>>>> >>>>> Thanks, >>>>> George >>>>> >>>>> -- >>>>> >>>>> --------------------------- >>>>> >>>>> George Kousiouris >>>>> Electrical and Computer Engineer >>>>> Division of Communications, >>>>> Electronics and Information Engineering >>>>> School of Electrical and Computer Engineering >>>>> Tel: +30 210 772 2546 >>>>> Mobile: +30 6939354121 >>>>> Fax: +30 210 772 2569 >>>>> Email: gkous...@mail.ntua.gr >>>>> Site: http://users.ntua.gr/gkousiou/ >>>>> >>>>> National Technical University of Athens >>>>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece >>>>> >>>>> >>>> >>> >>> >>> -- >>> >>> --------------------------- >>> >>> George Kousiouris >>> Electrical and Computer Engineer >>> Division of Communications, >>> Electronics and Information Engineering >>> School of Electrical and Computer Engineering >>> Tel: +30 210 772 2546 >>> Mobile: +30 6939354121 >>> Fax: +30 210 772 2569 >>> Email: gkous...@mail.ntua.gr >>> Site: http://users.ntua.gr/gkousiou/ >>> >>> National Technical University of Athens >>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece >>> >>
