I suggest testing without federation first. That is, running two separate
yarn/hdfs instances. Once that works properly, you can introduce federation in
your config.
Kihwal
From: xeonmailinglist <[email protected]>
To: [email protected]
Sent: Tuesday, March 3, 2015 4:32 AM
Subject: Questions about HDFS federation
Hi,
1. The image below shows 2 YARN instances using the same federation of
clouds. Is this possible?
2. I have configured in 2 hosts (hadooop-coc-1, and hadoop-coc-2) a
federation of HDFS. In the configuration, I have set a namespace in each host,
and a single data node (see image). The service is running properly. You can
check the output of the jps commands in [1]. The strange part is that, when I
list the files, they do not appear in hadoop-coc-2. You can check the output in
[2]. Why this happens?
[1]: jps output xubuntu@hadoop-coc-1:~/Programs/hadoop$ jps
21538 NameNode
21773 DataNode
xubuntu@hadoop-coc-2:~/Programs/hadoop$ jps
2342 NameNode
[2]: hdfs dfs -ls / output xubuntu@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs
-ls /
Java HotSpot(TM) Client VM warning: You have loaded library
/home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have
disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>',
or link it with '-z noexecstack'.
15/03/03 05:09:04 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Found 1 items
drwxr-xr-x - xubuntu supergroup 0 2015-03-03 04:47 /input1
xubuntu@hadoop-coc-2:~/Programs/hadoop$ hdfs dfs -ls /
Java HotSpot(TM) Client VM warning: You have loaded library
/home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have
disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>',
or link it with '-z noexecstack'.
15/03/03 05:09:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
--
--
