Try spark.driver.maxResultsSize =0
On Mon, 12 Sep 2022, 09:46 rajat kumar, wrote:
> Hello Users,
>
> My 2 tasks are running forever. One of them gave a java heap space error.
> I have 10 Joins , all tables are big. I understand this is data skewness.
> Apart from changes at code level , any prop
Hello Users,
My 2 tasks are running forever. One of them gave a java heap space error.
I have 10 Joins , all tables are big. I understand this is data skewness.
Apart from changes at code level , any property which can be used in Spark
Config?
I am using Spark2 hence AQE can not be used.
Thank
