I see that the pre-built distributions includes hive-shims-0.23 shaded in
spark-assembly jar (unlike when I make the distribution myself).
Does anyone knows what I should do to include the shims in my distribution?
On Thu, May 14, 2015 at 9:52 AM, Lior Chaga wrote:
> Ultimately it was PermGen o
Ultimately it was PermGen out of memory. I somehow missed it in the log
On Thu, May 14, 2015 at 9:24 AM, Lior Chaga wrote:
> After profiling with YourKit, I see there's an OutOfMemoryException in
> context SQLContext.applySchema. Again, it's a very small RDD. Each executor
> has 180GB RAM.
>
> O
After profiling with YourKit, I see there's an OutOfMemoryException in
context SQLContext.applySchema. Again, it's a very small RDD. Each executor
has 180GB RAM.
On Thu, May 14, 2015 at 8:53 AM, Lior Chaga wrote:
> Hi,
>
> Using spark sql with HiveContext. Spark version is 1.3.1
> When running l
Hi,
Using spark sql with HiveContext. Spark version is 1.3.1
When running local spark everything works fine. When running on spark
cluster I get ClassNotFoundError org.apache.hadoop.hive.shims.Hadoop23Shims.
This class belongs to hive-shims-0.23, and is a runtime dependency for
spark-hive:
[INFO]
