Can you submit a pull request to add that to the documentation?
On November 15, 2016 at 12:45:57 PM, Artur Sukhenko ( artur.sukhe...@gmail.com) wrote: Hello guys, When you enable ExternalShuffleService (spark-shuffle) in NodeManager, there are no suggestions of increasing NM heap size in Spark docs or anywhere else, shouldn't we include this in spark's documentation? I have seen NM take a lot of memory 5+ gb with default 1g, and in case of its GC pauses spark can become very slow when tasks are doing shuffle. I don't think users are aware of NM becoming bottleneck. Sincerely, Artur Sukhenko -- -- Artur Sukhenko
