Hello, I am facing problems in running fuse-dfs over hdfs. I came across this thread while searching for my problem: http://www.mail-archive.com/hdfs-user@hadoop.apache.org/msg00341.html OR http://search-hadoop.com/m/T1Bjv17q0eF1&subj=Re+Fuse+DFS
and it exactly mentions some of the symptoms I am looking at. To quote Eli, "fuse_impls_getattr.c connects via hdfsConnectAsUser so you should see a log (unless its returning from a case that doesn't print an error). Next step is to determine that you're actually reaching the code you modified by adding a syslog to the top of the function (need to make sure you're actually loading the libhdfs you've built vs an older one or another one installed on your system), and then determine which error case in that function you're seeing. It's strange that -d would cause that path to change." I get the error from fuse_impls_getattr.c that could not connect to my host and port. I checked the syslogs and only this error is printed there. No error is printed from hdfsConnectAsUser(). I even wrote down syslog functions in hdfsConnectAsUser, but it is not printed. I used the following commands to compile libhdfs and then fuse-dfs from $HADOOP_HOME: $ export LD_LIBRARY_PATH=$HADOOP_HOME/build/libhdfs:/usr/lib/jvm/java-1.5.0-sun: /usr/lib/jvm/java-1.5.0-sun-1.5.0.19/include:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/i386/client $ ant compile-c++-libhdfs -Dlibhdfs=1 $ ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1 -Djava5.home=/usr/lib/jvm/java-1.5.0-sun-1.5.0.19 $ mkdir /media/myDrive/newhdfs $ cd $HADOOP_HOME/build/contrib/fuse-dfs/ $ sudo sh fuse_dfs_wrapper.sh dfs://aastha-desktop:9000 /media/myDrive/newhdfs And when i do: $ sudo cat /var/log/syslog at the end, I get aastha-desktop fuse_dfs: ERROR: could not connect to aastha-desktop:9000, fuse_impls_getattr.c:37 EIO value is 5, which I understand implies an Input/output error. When i try to ls /media/myDrive/newhdfs, i get the input/output error. Also if i try to unmount newhdfs, it says that it is in use and cannot be unmounted. I even tried -d option, but that just runs into an infinite loop of connection retries and shows some socketconnection exception. I have been working with fuse-dfs and hdfs for more than a month now. Till two days back I could atleast create fuse-dfs instances and mount them using the wrapper script. I could also create files inside it, modify them, etc. and view the changes being updated in the nodes of the hadoop cluster. Since yesterday, I am trying to create a LUN image inside fuse-dfs (using dd if=/dev/zero of=... command), and check whether it is created properly on the cluster or not. Yesterday, a normal file system was working and when I tried to create a LUN, my syslog showed that an error was coming from fuse_impls_open.c. But now a normal file system instance is also not getting created and mounted properly. Following are my questions based on this observation: 1. Please tell me, why is the fuse_impls_getattr.c error is coming now and how can it be resolved. 2. Does ant compile-c++-libhdfs compile libhdfs? Will the changes that I make in source files in libhdfs be updated by the command? Since I can't see any syslog statements getting printed from hdfsConnectAsUser(), I am not sure if libhdfs has been recompiled. Also I cannot see any object files created inside libhdfs, so greater is my doubt. 3. How does libhdfs.so and libhdfs.so.0 come into picture in all this? We have provided links to them in build/libhdfs; how do they work? Any help with these issues would be highly appreciated. Thanks, Aastha. -- Aastha Mehta Intern, NetApp, Bangalore 4th year undergraduate, BITS Pilani E-mail: aasth...@gmail.com
