Re: Spark 2.1.0 and Shapeless

2017-01-31 Thread Koert Kuipers
shading at the fat jar level can work, however it means that in your unit tests where spark is a provided dependency you still can get errors because spark is using an incompatible (newer) shapeless version. the unit tests run with a single resolved shapeless after all. for example spark ships wit

Re: Spark 2.1.0 and Shapeless

2017-01-31 Thread Phil Wills
Are you not able to shade it when you're building your fat jar with something like https://github.com/sbt/sbt-assembly#shading? I would have thought doing the shading at the app level would be a bit less painful than doing it at the library level. Phil On Tue, 31 Jan 2017, 04:24 Timothy Chan, wr