Confirming that I am also hitting the same errors.
host: r3.8xlarge
configuration
spark.serializer org.apache.spark.serializer.KryoSerializer
spark.driver.memory 200g
spark.serializer.objectStreamReset 10
With spark.serializer.objectStreamReset set to 1, I ran a sample scala test
code which still seems to be crashing at the same place. If someone could
verify this independently, I would greatly appreciate it.
Scala Code:
--
import
Please decrease spark.serializer.objectStreamReset for your queries. The
default value is 100.
I logged SPARK-10787 for improvement.
Cheers
On Wed, Sep 23, 2015 at 6:59 PM, jluan wrote:
> I have been stuck on this problem for the last few days:
>
> I am attempting to run random forest from MLL
