Re: [ceph-users] hadoop on cephfs

2016-06-09 Thread Noah Watkins
On Sat, Apr 30, 2016 at 2:55 PM, Adam Tygart wrote: > Supposedly cephfs-hadoop worked and/or works on hadoop 2. I am in the > process of getting it working with cdh5.7.0 (based on hadoop 2.6.0). > I'm under the impression that it is/was working with 2.4.0 at some > point in time. > > At this very

Re: [ceph-users] hadoop on cephfs

2016-06-09 Thread Saverio Proto
You can also have Hadoop talking to the Rados Gateway (SWIFT API) so that the data is in Ceph instead of HDFS. I wrote this tutorial that might help: https://github.com/zioproto/hadoop-swift-tutorial Saverio 2016-04-30 23:55 GMT+02:00 Adam Tygart : > Supposedly cephfs-hadoop worked and/or works

Re: [ceph-users] hadoop on cephfs

2016-04-30 Thread Adam Tygart
Supposedly cephfs-hadoop worked and/or works on hadoop 2. I am in the process of getting it working with cdh5.7.0 (based on hadoop 2.6.0). I'm under the impression that it is/was working with 2.4.0 at some point in time. At this very moment, I can use all of the DFS tools built into hadoop to crea

Re: [ceph-users] hadoop on cephfs

2016-04-30 Thread Erik McCormick
I think what you are thinking of is the driver that was built to actually replace hdfs with rbd. As far as I know that thing had a very short lifespan on one version of hadoop. Very sad. As to what you proposed: 1) Don't use Cephfs in production pre-jewel. 2) running hdfs on top of ceph is a mas

Re: [ceph-users] hadoop on cephfs

2016-04-29 Thread Bill Sharer
Actually this guy is already a fan of Hadoop. I was just wondering whether anyone has been playing around with it on top of cephfs lately. It seems like the last round of papers were from around cuttlefish. On 04/28/2016 06:21 AM, Oliver Dzombic wrote: Hi, bad idea :-) Its of course nice a

Re: [ceph-users] hadoop on cephfs

2016-04-28 Thread Oliver Dzombic
Hi, bad idea :-) Its of course nice and important to drag developer towards a new/promising technology/software. But if the technology under the individual required specifications does not match, you will just risk to show this developer how worst this new/promising technology is. So you will j