Set spark.*.retained* configs to 0 when the UI is disabled?

2017-10-13 Thread Craig Ingram
I was recently debugging an OOM exception one of my coworkers was struggling with and found that `SQLListener._stageIdToStageMetrics` was the culprit. The UI was disabled in this case, but stats were still accumulating for jobs, stages, and tasks. The job my coworker was running had over 40k tasks

Re: Set spark.*.retained* configs to 0 when the UI is disabled?

2017-10-13 Thread Craig Ingram
Thanks for the feedback Marcelo! Are you referring to SPARK-20421 and SPARK-18085 ? If I can lend a hand in this, just let me know. -- Sent from: http://apache-spark-developers-list.100