https://spark.apache.org/docs/latest/sql-programming-guide.html#loading-data-programmatically
On Tue, Jul 21, 2015 at 4:06 PM, Ron Gonzalez <[email protected]> wrote: > Hi, > Question on using spark sql. > Can someone give an example for creating table from a directory > containing parquet files in HDFS instead of an actual parquet file? > > Thanks, > Ron > > On 07/21/2015 01:59 PM, Brandon White wrote: > >> A few questions about caching a table in Spark SQL. >> >> 1) Is there any difference between caching the dataframe and the table? >> >> df.cache() vs sqlContext.cacheTable("tableName") >> >> 2) Do you need to "warm up" the cache before seeing the performance >> benefits? Is the cache LRU? Do you need to run some queries on the table >> before it is cached in memory? >> >> 3) Is caching the table much faster than .saveAsTable? I am only seeing a >> 10 %- 20% performance increase. >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
