This issue definitely needs more investigation, but I just wanted to quickly check if anyone has run into this problem or has general guidance around it. We¹ve seen a performance degradation with a large heap on a simple map task (I.e. No shuffle). We¹ve seen the slowness starting around from 50GB heap. (I.e. spark.executor.memoty=50g) And, when we checked the CPU usage, there were just a lot of GCs going on.
Has anyone seen a similar problem? Thanks, Mingyu
smime.p7s
Description: S/MIME cryptographic signature
