Re: Hive on Spark not working

2016-11-28 Thread Furcy Pin
ClassNotFoundException generally means that jars are missing from your class path. You probably need to link the spark jar to $HIVE_HOME/lib https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started#HiveonSpark:GettingStarted-ConfiguringHive On Tue, Nov 29, 2016 at 2:03 AM

Hive on Spark not working

2016-11-28 Thread Joaquin Alzola
Hi Guys No matter what I do that when I execute "select count(*) from employee" I get the following output on the logs: It is quiet funny because if I put hive.execution.engine=mr the output is correct. If I put hive.execution.engine=spark then I get the bellow errors. If I do the search directl

Re: Question about partition pruning when there's a type mismatch

2016-11-28 Thread Gopal Vijayaraghavan
> I'm wondering why Hive tries to scan all partitions when the quotes are > omitted. Without the quotes, shouldn't 2016-11-28-00 get evaluated as an > arithmetic expression, then get cast to a string, and then partitioning > pruning still occur? The order of evaluation is different - String =

Re: hive2.1.0 timestamp

2016-11-28 Thread Jason Dere
Looks like this might be HIVE-15291. From: ? ? Sent: Wednesday, November 23, 2016 6:59 PM To: user@hive.apache.org Subject: hive2.1.0 timestamp Hi: I am using hive2.1 and hava a problem.I have a table 'xp' which contains a timestamp field 'createdate'.When I

Question about partition pruning when there's a type mismatch

2016-11-28 Thread Anthony Hsu
Hi, Some of our Hive tables are partitioned on a STRING column "datepartition" whose format is "-mm-dd-hh". Usually, when users want to read only a specific day of data, they'll specify WHERE datepartition = '2016-11-28-00', for example. However, sometimes users accidentally forget the quotes

Re: Need error logging advice on batch processing

2016-11-28 Thread neelima g
Brotanek, *hive -f file.txt hiveconf hive.cli.errors.ignore=true 2&> file.log* *and then grep for keywords *Exception** Have you ever tried capturing what all hive queries failed i.e entire query.I am looking for line number of query in file or queries failed? Errors are in the format like :

RE: Need error logging advice on batch processing

2016-11-28 Thread Brotanek, Jan
Hello, you can log like this: hive -f file.txt hiveconf hive.cli.errors.ignore=true 2&> file.log and then grep for keywords *Exception* From: neelima g [mailto:nling...@gmail.com] Sent: pondělí 28. listopadu 2016 16:15 To: user@hive.apache.org Subject: Need error logging advice on batch process

Need error logging advice on batch processing

2016-11-28 Thread neelima g
Hi, I want to run batch mode task like and i am using *hiveconf hive.cli.errors.ignore=true* hive -f file.txt hiveconf hive.cli.errors.ignore=true And my requirement is find what all queries failed after execution. Is that possible ? And i have create statements in the queries..it is DDL.

Re: Some Hive on Tez queries don't finish

2016-11-28 Thread Rajesh Balamohan
Are there are any exceptions seen in the app logs (you can ignore the Interrupted exceptions in the logs as you killed the job). It would be helpful if you can share the app logs. ~Rajesh.B On Mon, Nov 28, 2016 at 2:53 PM, Premal Shah wrote: > Hi, > We've been running Hive 2.0.1 on Tez 0.8.4 fo

Some Hive on Tez queries don't finish

2016-11-28 Thread Premal Shah
Hi, We've been running Hive 2.0.1 on Tez 0.8.4 for a few weeks now. Most queries that we run work. However some queries that go over millions to billions of rows don't finish using Tez as the execution engine. Here's an example of a simple query that does not finish select count(distinct external