Yeah, I had brought that up a while back, but didn't get agreement on
removing the stub. Seems to be an intermittent problem. You can just add
an exclude:
mergeStrategy in assembly := {
case PathList("org", "apache", "spark", "unused", "UnusedStubClass.class")
=> MergeStrategy.first
case x => (mergeStrategy in assembly).value(x)
}
On Mon, Jul 13, 2015 at 6:55 AM, Luis Ángel Vicente Sánchez <
[email protected]> wrote:
> I have just upgraded to spark 1.4.0 and it seems that
> spark-streaming-kafka has a dependency on org.spark-project.spark unused
> 1.0.0 but it also embeds that jar in its artifact, causing a problem while
> creating a fatjar.
>
> This is the error:
>
> [Step 1/1] (*:assembly) deduplicate: different file contents found in the
>> following:
>>
>> /data/system/sbt_ivy/cache/org.apache.spark/spark-streaming-kafka_2.10/jars/spark-streaming-kafka_2.10-1.4.0.jar:org/apache/spark/unused/UnusedStubClass.class
>>
>> /data/system/sbt_ivy/cache/org.spark-project.spark/unused/jars/unused-1.0.0.jar:org/apache/spark/unused/UnusedStubClass.class
>
>
