@Sean perhaps I could leverage this when this
http://openjdk.java.net/jeps/261 becomes available.
On Fri, Dec 16, 2016 at 4:05 AM, Steve Loughran <ste...@hortonworks.com>
wrote:
> FWIW, although the underlying Hadoop declared guava dependency is pretty
> low, everything in org.apache.hadoop is set up to run against later
> versions. It just sticks with the old one to avoid breaking anything
> donwstream which does expect a low version number. See HADOOP-10101 for the
> ongoing pain there —and complain on there if you do find something in the
> Hadoop layer which can't handle later guava versions.
>
>
>
>
> On 16 Dec 2016, at 11:07, Sean Owen <so...@cloudera.com> wrote:
>
> Yes, that's the problem. Guava isn't generally mutually compatible across
> more than a couple major releases. You may have to hunt for a version that
> happens to have the functionality that both dependencies want, and hope
> that exists. Spark should shade Guava at this point but doesn't mean that
> you won't hit this problem from transitive dependencies.
>
> On Fri, Dec 16, 2016 at 11:05 AM kant kodali <kanth...@gmail.com> wrote:
>
>> I replaced *guava-14.0.1.jar* with *guava-19.0.jar in *SPARK_HOME/jars
>> and seem to work ok but I am not sure if it is the right thing to do. My
>> fear is that if Spark uses features from Guava that are only present in
>> 14.0.1 but not in 19.0 I guess my app will break.
>>
>>
>>
>> On Fri, Dec 16, 2016 at 2:22 AM, kant kodali <kanth...@gmail.com> wrote:
>>
>> Hi Guys,
>>
>> Here is the simplified version of my problem. I have the following
>> problem and I new to gradle
>>
>>
>> dependencies {
>> compile group: 'org.apache.spark', name: 'spark-core_2.11', version:
>> '2.0.2'
>> compile group: 'com.github.brainlag', name: 'nsq-client', version:
>> '1.0.0.RC2'
>> }
>>
>>
>> I took out the other dependencies for simplicity. The problem here
>> is spark-core_2.11 uses com.google.guava:14.0.1 and nsq-client uses
>> com.google.guava:19.0 so when I submit my fat uber jar using spark-submit I
>> get the following error
>>
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> com.google.common.collect.Sets.newConcurrentHashSet()Ljava/util/Set;
>> at com.github.brainlag.nsq.NSQProducer.(NSQProducer.java:22)
>> at com.hello.streamprocessing.app.SparkDriver2.main(SparkDriver2.java:37)
>>
>>
>> any help would be great. Also if you need more description you can find
>> it here
>> <http://stackoverflow.com/questions/41003416/how-to-solve-nomethoderror-that-arises-due-to-using-a-same-library-with-two-diff>
>>
>> Thanks!
>>
>>
>>
>
