; user@spark.apache.org
Subject: Re: configure number of cached partition in memory on SparkSQL
Hi Judy,
In the case of HadoopRDD and NewHadoopRDD, partition number is actually decided
by the InputFormat used. And spark.sql.inMemoryColumnarStorage.batchSize is not
related to partition number, it
Hi Judy,
In the case of |HadoopRDD| and |NewHadoopRDD|, partition number is
actually decided by the |InputFormat| used. And
|spark.sql.inMemoryColumnarStorage.batchSize| is not related to
partition number, it controls the in-memory columnar batch size within a
single partition.
Also, what d
