> I understood that caching a table pegged the RDD partitions into the
memory of the executors holding the partition.
Your understanding is correct. Nothing to worry on the driver side while
creating the temp view.
On Sun, Feb 16, 2025, 10:47 PM Mich Talebzadeh
wrote:
> Ok let us look at this
>
where storage will always have more
priority than execution and will never be released to execution.
Regards,
Subhasis Mukherjee
From: Sreyan Chakravarty
Sent: Wednesday, August 14, 2024 9:00:45 PM
To: user@spark.apache.org
Subject: Need help understanding tuning
Regarding making spark writer fast part, If you are (or can be) on Databricks,
check this out. It is just out of the oven at Databricks.
https://www.databricks.com/blog/announcing-general-availability-liquid-clustering?utm_source=bambu&utm_medium=social&utm_campaign=advocacy&blaid=6087618
