I am running into an issues connecting hadoop to my ceph cluster and I'm sure I am missing something but can't figure it out. I have a Ceph cluster with MDS running fine and I can do a basic mount perfectly normal. I have hadoop fs -ls with basic file:/// working well.
Info: ceph cluster version 0.61.7 Ubuntu Server 13.04 x86_64 hadoop 1.2.1-1 deb install (stable now I did try 1.1.2 same issue) libcephfs-java both hadoop-cephfs.jar and libcephfs.jar show up in "hadoop classpath" libcephfs-jni with symlink trick /usr/share/hadoop/lib/native/Linux-amd64-64 listed here http://thread.gmane.org/gmane.comp.file-systems.ceph.user/1788/focus=1806 and the LD_LIBRARY_PATH in hadoop-env.sh When I try to setup the ceph mount within Hadoop I get an exception $ hadoop fs -ls Exception in thread "main" java.lang.NoClassDefFoundError: com/ceph/fs/CephPoolException at org.apache.hadoop.fs.ceph.CephFileSystem.initialize(CephFileSystem.java:96) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1446) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:67) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1464) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:263) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:124) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:247) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187) at org.apache.hadoop.fs.FsShell.ls(FsShell.java:583) at org.apache.hadoop.fs.FsShell.run(FsShell.java:1812) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.hadoop.fs.FsShell.main(FsShell.java:1916) Caused by: java.lang.ClassNotFoundException: com.ceph.fs.CephPoolException Followed the tutorial here http://ceph.com/docs/next/cephfs/hadoop/ core-site.xml settings ... <property> <name>fs.ceph.impl</name> <value>org.apache.hadoop.fs.ceph.CephFileSystem</value> </property> <property> <name>fs.default.name</name> <value>ceph://192.168.1.11:6789</value> </property> <property> <name>ceph.data.pools</name> <value>hadoop1</value> </property> <property> <name>ceph.auth.id</name> <value>admin</value> </property> <property> <name>ceph.auth.keyfile</name> <value>/etc/ceph/admin.secret</value> </property> Any Help Appreciated
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
