Re: Handling Hive Table With large number of rows

2016-02-07 Thread Meetu Maltiar
Thanks Jörn, We have to construct an XML on HDFS location from couple of Hive tables and they join on one key. The data in both tables we have to join is large. Was wondering for the right approach. XML creation will also be tricky as we cannot hold objects in memory. Old Spark 1.2.1 is a bummer,

Re: Handling Hive Table With large number of rows

2016-02-07 Thread Jörn Franke
Can you provide more details? Your use case does not sound you need Spark. Your version is anyway too old. It does not make sense to develop now with 1.2.1 . There is no "project limitation" that is able to justify this. > On 08 Feb 2016, at 06:48, Meetu Maltiar wrote: > > Hi, > > I am workin