Re: Spark SQL is not returning records for HIVE transactional tables on HDP

2016-03-13 Thread Gopal Vijayaraghavan
> We are using for Spark SQL : Does SparkSQL support Transactional tables? I thought Transactional tables needed a dead-lock proof LockManager, which was a hive-specific feature? Cheers, Gopal

Re: Hive on Spark performance

2016-03-13 Thread Mich Talebzadeh
Depending on the version of Hive on Spark engine. As far as I am aware the latest version of Hive that I am using (Hive 2) has improvements compared to the previous versions of Hive (0.14,1.2.1) on Spark engine. As of today I have managed to use Hive 2.0 on Spark version 1.3.1. So it is not the l

Hive on Spark performance

2016-03-13 Thread sjayatheertha
Just curious if you could share your experience on the performance of spark in your company? How much data do you process? And what's the latency you are getting with spark engine? Vidya

Re: how to create an array from two columns?

2016-03-13 Thread Rex X
Very nice! Thank you, Chandeep. On Sat, Mar 12, 2016 at 8:20 PM, Chandeep Singh wrote: > Not sure if there is a function for that but I wrote a UDF to do so - > https://github.com/chandeepsingh/Hive-UDFs > > hive> ADD JAR hive-udfs-1.0-uber.jar; > Added [hive-udfs-1.0-uber.jar] to class path >

Re: Spark SQL is not returning records for HIVE transactional tables on HDP

2016-03-13 Thread @Sanjiv Singh
Hi All, We are using for Spark SQL : - Hive :1.2.1 - Spark : 1.3.1 - Hadoop :2.7.1 Let me know if needs other details to debug the issue. Regards Sanjiv Singh Mob : +091 9990-447-339 On Sun, Mar 13, 2016 at 1:07 AM, Mich Talebzadeh wrote: > Hi, > > Thanks for the input. I use Hive

Re: How to rename a hive table without changing location?

2016-03-13 Thread Mich Talebzadeh
Hi, ALTER TABLE ... RENAME just renames table and its files leaving the location in the same database to save work and space. Other alternative is going to another database (which basically means another sub-directory of warehouse and create the table with CREATE TABLE AS SELECT * from TABLE ..