Re: Hive on spark table caching

2015-12-02 Thread Xuefu Zhang
Depending on the query, Hive on Spark does implicitly cache datasets (not necessarily the input tables) for performance benefits. Such queries include multi-insert, self-join, self-union, etc. However, no caching happens across queries at this time, which may be improved in the future. Thanks, Xue

RE: Hive on spark table caching

2015-12-02 Thread Mich Talebzadeh
onsibility. From: Udit Mehta [mailto:ume...@groupon.com] Sent: 02 December 2015 23:43 To: user@hive.apache.org Subject: Re: Hive on spark table caching Im using Spark 1.3 with Hive 1.2.1. I dont mind using a version of Spark higher than that but I read somewhere that 1.3 is the version of

Re: Hive on spark table caching

2015-12-02 Thread Udit Mehta
Im using Spark 1.3 with Hive 1.2.1. I dont mind using a version of Spark higher than that but I read somewhere that 1.3 is the version of Spark currently supported by Hive. Can I use Spark 1.4 or 1.5 with Hive 1.2.1? On Wed, Dec 2, 2015 at 3:19 PM, Mich Talebzadeh wrote: > Hi, > > > > Which vers

RE: Hive on spark table caching

2015-12-02 Thread Mich Talebzadeh
Hi, Which version of spark are you using please? Mich Talebzadeh Sybase ASE 15 Gold Medal Award 2008 A Winning Strategy: Running the most Critical Financial Data on ASE 15 http://login.sybase.com/file