Looking at the lib folder revealed the problem. The lib folder on one of
the nodes had libraries for both hadoop 1 and 2. I’m not sure how I ended
up with that but it must have happened while I was copying the dependency
jars to each node. I removed all the jars and started with a fresh copy
and re
Okay, then, my previous statement is false. From the stack trace, it seems
that Flink is using an older Hadoop version. The DFSClient and RPC classes
look different in Hadoop 2.6.0.
Max checked already for some environment variables. Is $CLASSPATH set? Did
you install Hadoop only by downloading the
Hi Robert,
On the ubuntu host, the port 9000 is up:
ubuntu@ubuntu-171:~$ netstat -tulpn
tcp0 0 10.13.182.171:9000 0.0.0.0:* LISTEN
8849/java
And the process using this port is:
ubuntu@ubuntu-171:~$ ps -ef | grep 8849
ubuntu8849 1 0 2015 ?
Your Flink installation has Hadoop 2.6.0 included, on the other machine,
there is a Hadoop version installed, which is most likely a 1.x or even a
0.x version.
Are you sure that the host "ubuntu-171" has the ip 10.13.182.171 and that
the hadoop installation in the "/home/ubuntu/Documents/hadoop-2.6
Hi Max,
Both commands return nothing. Those variables aren’t set.
The only software I installed on these machines is Flink and Java.
-Ali
On 2015-12-28, 6:42 AM, "Maximilian Michels" wrote:
>Hi Ali,
>
>The warning about the native Hadoop libraries is nothing to worry
>about. The native module
Hi Ali,
The warning about the native Hadoop libraries is nothing to worry
about. The native modules are platform-optimized modules which may be
used to improve performance. They are not necessary for Hadoop to
function correctly.
The exception message implies that you are using is a very old vers
Hi Max,
I have the same output for the Task Manager:
11:25:04,274 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Hadoop version: 2.6.0
I do get this line at the beginning of both job and task manager log files:
11:25:04,100 WARN org.apache.hadoop.util.NativeCodeLoader
-
Hi Ali,
Could you please also post the Hadoop version output of the task
manager log files? It looks like the task managers are running a
different Hadoop version.
Thanks,
Max
On Tue, Dec 22, 2015 at 4:28 PM, Kashmar, Ali wrote:
> Hi Robert,
>
> I found the version in the job manager log file:
Hi Robert,
I found the version in the job manager log file:
17:33:49,636 INFO org.apache.flink.runtime.jobmanager.JobManager
- Hadoop version: 2.6.0
But the Hadoop installation I have is saying this:
ubuntu@ubuntu-171:~/Documents/hadoop-2.6.0$ bin/hadoop version
Hadoop 2.6.0
Subversion
Hi Ali,
the TaskManagers and the JobManager is logging the Hadoop version on
startup.
On Tue, Dec 22, 2015 at 4:10 PM, Kashmar, Ali wrote:
> Hello,
>
> I’m trying to use HDFS as store for Flink checkpoints so I downloaded the
> Hadoop 2.6.0/Scala 2.10 version of Flink and installed it. I also
>
Hello,
I’m trying to use HDFS as store for Flink checkpoints so I downloaded the
Hadoop 2.6.0/Scala 2.10 version of Flink and installed it. I also downloaded
Hadoop 2.6.0 separately from the Hadoop website and set up HDFS on a separate
machine. When I start Flink I get the following error:
17:
11 matches
Mail list logo
