Hi Steffen, this looks like a Guava version mismatch to me. Are you running exactly the same program on your local machine or did you add dependencies to run it on the cluster (e.g. Kinesis). Maybe Kinesis and Elasticsearch are using different Guava versions?
Best, Fabian 2016-09-01 10:45 GMT+02:00 Hausmann, Steffen <[email protected]>: > Hi there, > > I’m running a flink program that reads from a Kinesis stream and > eventually writes to an Elasticsearch2 sink. When I’m running the program > locally from the IDE, everything seems to work fine, but when I’m executing > the same program on an EMR cluster with Yarn, a NoClassDefFoundError occurs: > > java.lang.NoSuchMethodError: com.google.common.util.concurr > ent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor; > > at org.elasticsearch.threadpool.ThreadPool.<clinit>(ThreadPool. > java:190) > > at org.elasticsearch.client.transport.TransportClient$Builder. > build(TransportClient.java:133) > > at org.apache.flink.streaming.connectors.elasticsearch2.Elastic > searchSink.open(ElasticsearchSink.java:164) > > at org.apache.flink.api.common.functions.util.FunctionUtils.ope > nFunction(FunctionUtils.java:38) > > at org.apache.flink.streaming.api.operators.AbstractUdfStreamOp > erator.open(AbstractUdfStreamOperator.java:91) > > at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllO > perators(StreamTask.java:376) > > at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke( > StreamTask.java:256) > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584) > > at java.lang.Thread.run(Thread.java:745) > > I’ve installed flink on an EMR cluster from the binary distribution > flink-1.1.1-bin-hadoop27-scala_2.10.tgz. I build the jar file locally > with mvn clean package(I’ve attached the pom.xml for reference) and copy it > to the cluster for execution. There is a thread on this list that seems to > be related, but I’m afraid I couldn’t draw any conclusions from it: > > http://apache-flink-user-mailing-list-archive.2336050.n4. > nabble.com/classpath-issue-on-yarn-tt6442.html#none > > > Any idea, what’s wrong? > > Thanks, > > Steffen > >
