Re: [ceph-users] how to use hadoop when system is CephFS

2013-10-19 Thread Noah Watkins
On Fri, Oct 18, 2013 at 7:31 PM, 鹏 wrote: > hi Noah > That is a stupid mistake which I make! the reason is start-all.sh not > start the datanode! so I use the shell start-mapr.sh to start it! > by the way! is the ceph repalce HDFS is respace the name node??? > thank you ,Noah! > peng There is n

Re: [ceph-users] how to use hadoop when system is CephFS

2013-10-18 Thread
hi Noah That is a stupid mistake which I make! the reason is start-all.sh not start the datanode! so I use the shell start-mapr.sh to start it! by the way! is the ceph repalce HDFS is respace the name node??? thank you ,Noah! peng At 2013-10-18 22:27:56,"Noah Watkins" wrote: Kai, It looks lik

Re: [ceph-users] how to use hadoop when system is CephFS

2013-10-18 Thread log1024
Hi Peng, Noah Thanks for your help. Now the hadoop & ceph are working now! It takes 11m59.464s to finish MrBenchmark x 50 times. Cheers! Kai At 2013-10-18 06:27:56,"Noah Watkins" wrote: Kai, It looks like libcephfs-java (the CephFS Java bindings) are not in your classpath. Where did

Re: [ceph-users] how to use hadoop when system is CephFS

2013-10-18 Thread Noah Watkins
Peng, I'm glad you were able to get it working. You'll have to provide some more information to start debugging why it is slow. How is your Ceph cluster configured? Also, have a look at the jobtracker statistics and see if any tasks are failing. On Thu, Oct 17, 2013 at 8:17 PM, 鹏 wrote: > ** >

Re: [ceph-users] how to use hadoop when system is CephFS

2013-10-18 Thread Noah Watkins
Kai, It looks like libcephfs-java (the CephFS Java bindings) are not in your classpath. Where did you install them? -Noah On Thu, Oct 17, 2013 at 11:30 PM, log1024 wrote: > Hi Peng > The conf in my cluster is almost the same with yours, but when i run > #bin/hadoop fs -ls / > It failed with:

Re: [ceph-users] how to use hadoop when system is CephFS

2013-10-17 Thread log1024
Hi Peng The conf in my cluster is almost the same with yours, but when i run #bin/hadoop fs -ls / It failed with: Exception in thread "main" java.lang.NoClassDefFoundError: com/ceph/fs/CephFileAlreadyExistsException at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:2

[ceph-users] how to use hadoop when system is CephFS

2013-10-17 Thread
| Hi all ! I think I suceed using cephFS with hadoop, but I still face lost question for that I am a newer in hadoop and ceph! first, my ceph version is 0.62. java is 1.6.0_45 .hadoop is 1.1.2 I think suceed because: # hadoop fs -ls / drwxrwxrwx - root 0 2013-10-16 10:57 /