Re: Facing memory leak with Pyarrow enabled and toPandas()

Hi Can you please mention the spark version, give us the code for setting up spark session, and the operation you are talking about? It will be good to know the amount of memory that your system has as well and number of executors you are using per system In general I have faced issues when doing g

Facing memory leak with Pyarrow enabled and toPandas()

Hi, I am facing this issue while using toPandas() and Pyarrow simultaneously. pandas - toPandas() giving memory leak error with PySpark arrow enabled - Stack Overflow