Re: NoClassDefFoundError with ElasticsearchSink on Yarn

2016-09-09 Thread Fabian Hueske
+1 I ran into that issue as well. Would be great to have that in the docs! 2016-09-09 11:49 GMT+02:00 Robert Metzger : > Hi Steffen, > > I think it would be good to add it to the documentation. > Would you like to open a pull request? > > > Regards, > Robert > > > On Mon, Sep 5, 2016 at 10:26 PM,

Re: NoClassDefFoundError with ElasticsearchSink on Yarn

2016-09-09 Thread Robert Metzger
Hi Steffen, I think it would be good to add it to the documentation. Would you like to open a pull request? Regards, Robert On Mon, Sep 5, 2016 at 10:26 PM, Steffen Hausmann < stef...@hausmann-family.de> wrote: > Thanks Aris for your explanation! > > A guava version mismatch was indeed the pr

Re: NoClassDefFoundError with ElasticsearchSink on Yarn

2016-09-05 Thread Steffen Hausmann
Thanks Aris for your explanation! A guava version mismatch was indeed the problem. But in addition to shading the guava dependencies, I encountered another issue caused by conflicting files in META-INF/services: RemoteTransportException[[Failed to deserialize response of type [org.elasticse

Re: NoClassDefFoundError with ElasticsearchSink on Yarn

2016-09-01 Thread aris kol
Classic problem of every uber-jar containing Hadoop dependencies and being deployed on Yarn. What actually happens is that some Hadoop dependency relies on an old version of guava (11 in this case), which doesn't have the method. You may have assembled your fat-jar properly, but because Hadoop

Re: NoClassDefFoundError with ElasticsearchSink on Yarn

2016-09-01 Thread Fabian Hueske
Hi Steffen, this looks like a Guava version mismatch to me. Are you running exactly the same program on your local machine or did you add dependencies to run it on the cluster (e.g. Kinesis). Maybe Kinesis and Elasticsearch are using different Guava versions? Best, Fabian 2016-09-01 10:45 GMT+02