Hi, I'm querying a big table using Spark SQL. I see very long GC time in some stages. I wonder if I can improve it by tuning the storage parameter.
The question is: the schemaRDD has been cached with "cacheTable()" function. So is the cached schemaRDD part of memory storage controlled by the "spark.storage.memoryFraction" parameter? Thanks! --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
