Run times for Spark 1.6.2 compared to 2.1.0?

2016-07-27 Thread Colin Beckingham
I have a project which runs fine in both Spark 1.6.2 and 2.1.0. It calculates a logistic model using MLlib. I compiled the 2.1 today from source and took the version 1 as a precompiled version with Hadoop. The odd thing is that on 1.6.2 the project produces an answer in 350 sec and the 2.1.0 ta

Re: Run times for Spark 1.6.2 compared to 2.1.0?

2016-07-28 Thread Colin Beckingham
On 27/07/16 16:31, Colin Beckingham wrote: I have a project which runs fine in both Spark 1.6.2 and 2.1.0. It calculates a logistic model using MLlib. I compiled the 2.1 today from source and took the version 1 as a precompiled version with Hadoop. The odd thing is that on 1.6.2 the project