Hi,
I want to copy data between hdfs in different hosts using webhdfs in
hadoop MRv2. The destination host has the user and group as root and
supergroup.
WorkGroup0010:~/# hdfs dfs -ls /
14/04/12 16:42:18 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes
where applicable
Found 3 items
drwxrwx--- - root supergroup 0 2014-04-12 16:25 /root
drwxr-xr-x - root supergroup 0 2014-04-12 16:25 /tmp
drwxrwxrwx - root supergroup 0 2014-04-12 16:39 /wiki
I defined in the core-site.xml these params for the user and the group.
WorkGroup0010:~/# cat ../hadoop/etc/hadoop/core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property> <name>fs.default.name</name>
<value>hdfs://172.16.100.5:9000</value> </property>
<property> <name>hadoop.tmp.dir</name>
<value>/tmp/hadoop-temp</value> </property>
<property><name>hadoop.proxyuser.root.hosts</name><value>*</value></property>
<property><name>hadoop.proxyuser.supergroup.groups</name><value>*</value></property>
<-- I am trying to use with thisgroup definition
<!--
property><name>hadoop.proxyuser.root.groups</name><value>*</value></property
--> <-- or with this group
</configuration>
When I copy data
WorkGroup0000:~/# /root/Programs/hadoop/bin/hadoop distcp
webhdfs://WorkGroup0000:50070/wiki webhdfs://WorkGroup0010:50070/wiki
I get the error on the destination as:
2014-04-12 16:37:06,720 ERROR
org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:root (auth:SIMPLE)
cause:java.io.FileNotFoundException: File does not exist:
/wiki/wiki/wv92.txt
1 - I don't understand why this is happening. I should have been able to
copy data. Why I can't copy data between hdfs?
2 - Is it possible to turn off the privilege rules?
--
Thanks,
