Andy, We did run HBase on Ceph but were not able to get a stable performance for sustained periods. It would give good throughput for a while and the client would crash soon thereafter. However, this was more than a year ago and a re-evaluation should be done before commenting further.
-Amandeep On Mon, Aug 29, 2011 at 8:09 PM, Alex Nelson <ajnel...@cs.ucsc.edu> wrote: > Hi Andy, > > When I ran Hadoop on Ceph, I was able to complete a run of Terasort. We > didn't run HBase. > > After I submitted a patch to the Ceph mailing list, Noah Watkins (CC'd) > took over developing the file system class in Hadoop's code base. I don't > know what's become of the work since then. The JIRA is here: > https://issues.apache.org/jira/browse/HADOOP-6779 > > The last I heard, which was a while ago, there was a snag with the URI > scheme (ceph:// vs. file:// or something similar) which had a subtle but > challenging complexity. Noah is more able to answer questions at this > point. He has posted Java RADOS bindings to the Ceph mailing list recently, > but I'm not sure what he has been up to aside from that. > > --Alex > > > On Aug 27, 2011, at 02:07 , Andrew Purtell wrote: > > > Hi Alex, > > > >> UC Santa Cruz contributed a ;login: article describing replacing HDFS > with > >> Ceph. (I was one of the authors.) One of the key architectural > advantages of > >> Ceph over HDFS is that Ceph distributes its metadata service over > multiple > >> metadata servers. > > > > Were you able to maintain stable operation with Ceph at a high load? Did > you try running HBase? > > > > Best regards, > > > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > > > > > > ----- Original Message ----- > >> From: Alex Nelson <ajnel...@cs.ucsc.edu> > >> To: George Kousiouris <gkous...@mail.ntua.gr> > >> Cc: common-dev@hadoop.apache.org; common-u...@hadoop.apache.org; Carlos > Maltzahn <carl...@soe.ucsc.edu> > >> Sent: Friday, August 26, 2011 2:44 AM > >> Subject: Re: Any related paper on how to resolve hadoop SPOF issue? > >> > >> Hi George, > >> > >> UC Santa Cruz contributed a ;login: article describing replacing HDFS > with > >> Ceph. (I was one of the authors.) One of the key architectural > advantages of > >> Ceph over HDFS is that Ceph distributes its metadata service over > multiple > >> metadata servers. > >> > >> I hope that helps. > >> > >> --Alex > >> > >> > >> On Aug 25, 2011, at 02:54 , George Kousiouris wrote: > >> > >>> > >>> Hi, > >>> > >>> many thanks!! > >>> > >>> I also found this one, which has some related work also with other > efforts, > >> in case it is helpful for someone else: > >>> > >> > https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1 > >>> > >>> > >>> BR, > >>> George > >>> > >>> On 8/25/2011 12:46 PM, Nan Zhu wrote: > >>>> Hope it helps > >>>> > >>>> http://www.springerlink.com/content/h17r882710314147/ > >>>> > >>>> Best, > >>>> > >>>> Nan > >>>> > >>>> On Thu, Aug 25, 2011 at 5:43 PM, George > >> Kousiouris<gkous...@mail.ntua.gr>wrote: > >>>> > >>>>> Hi guys, > >>>>> > >>>>> We are currently in the process of writing a paper regarding hadoop > >> and we > >>>>> would like to reference any attempt to remove the single point of > >> failure of > >>>>> the Namenode. We have found in various presentations some efforts > >> (like > >>>>> dividing the namespace between more than one namenodes if i > >> remember > >>>>> correctly) but the search for a concrete paper on the issue came to > >> nothing. > >>>>> > >>>>> Is anyone aware or has participated in such an effort? > >>>>> > >>>>> Thanks, > >>>>> George > >>>>> > >>>>> -- > >>>>> > >>>>> --------------------------- > >>>>> > >>>>> George Kousiouris > >>>>> Electrical and Computer Engineer > >>>>> Division of Communications, > >>>>> Electronics and Information Engineering > >>>>> School of Electrical and Computer Engineering > >>>>> Tel: +30 210 772 2546 > >>>>> Mobile: +30 6939354121 > >>>>> Fax: +30 210 772 2569 > >>>>> Email: gkous...@mail.ntua.gr > >>>>> Site: http://users.ntua.gr/gkousiou/ > >>>>> > >>>>> National Technical University of Athens > >>>>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece > >>>>> > >>>>> > >>>> > >>> > >>> > >>> -- > >>> > >>> --------------------------- > >>> > >>> George Kousiouris > >>> Electrical and Computer Engineer > >>> Division of Communications, > >>> Electronics and Information Engineering > >>> School of Electrical and Computer Engineering > >>> Tel: +30 210 772 2546 > >>> Mobile: +30 6939354121 > >>> Fax: +30 210 772 2569 > >>> Email: gkous...@mail.ntua.gr > >>> Site: http://users.ntua.gr/gkousiou/ > >>> > >>> National Technical University of Athens > >>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece > >>> > >> > >
