Re: Hash table in map join - Hive

Hi, I got a chance to re-run this query today and it does auto-reduce the new CUSTOM_EDGE join as well. However it does it too early, before it has got enough information about both sides of the join. TPC-DS Query79 at 1Tb scale generates a CUSTOM_EDGE between the ms alias and customer tables.

Yarn Application ID for Hive query

Hi, I am using the below python program to run a hive query. How can I get the Yarn application ID using the python program for the hive query execution. import pyhs2 with pyhs2.connect(host='abc.sac.com', port=1, authMechanism="PLAIN", user='ami

Re: msck repair table and hive v2.1.0

Hi Rajesh, sure. i'll give that setting a try. thanks. re: s3 vs. hdfs. indeed. I figured i'd eliminate the s3 angle when posting here given the msck repair table failed in both cases. but yeah my real use case is using s3. ok. just tried that setting and got a slightly different stack trace but

答复: Hive failed with "No files matching path"

Why not output for "echo $sql;"? 发件人: gm yu 发送时间: 2016年7月15日 9:59:39 收件人: user@hive.apache.org 主题: Hive failed with "No files matching path" Hello I have a question about Hive(2.0.0) and HDFS(2.7.2). I met many Hive failure with "No files matching path". The sc

Hive failed with "No files matching path"

Hello I have a question about Hive(2.0.0) and HDFS(2.7.2). I met many Hive failure with "No files matching path". The script like: hadoop fs -test -e $hdfspath if [ $? -ne 0 ]; then echo "Directory not exists!" hadoop fs -mkdir $hdfspath else echo "Directory exists!" fi hadoop distcp

Re: msck repair table and hive v2.1.0

Hi Stephen, Can you try by turning off multi-threaded approach by setting "hive.mv.files.thread=0"? You mentioned that your tables tables are in s3, but the external table created was pointing to HDFS. Was that intentional? ~Rajesh.B On Fri, Jul 15, 2016 at 6:58 AM, Stephen Sprague wrote: > i

Re: msck repair table and hive v2.1.0

in the meantime given my tables are in s3 i've written a utility to do a 'aws s3 ls' on the bucket and folder in question, change the folder syntax to partition syntax and then issued my own 'alter table ... add partition' for each partition. so essentially it does what msck repair tables does bu

Re: Want to be one contributor

https://cwiki.apache.org/confluence/display/Hive/Home#Home-ResourcesforContributors is a good place to start. Welcome to Hive. Alan. > On Jul 14, 2016, at 16:01, Alpesh Patel wrote: > > Hi Guys, > > I am part of this group since 1 year. Just an audience and now want to be > contributor in

Want to be one contributor

Hi Guys, I am part of this group since 1 year. Just an audience and now want to be contributor in Hive code base. Can you please guide me like how can i be contributor ? Is there any wiki which i can read for this ? Rgds, Alpesh

Re: A dedicated Web UI interface for Hive

What do you want it to do? There are at least two web interfaces I can think of. On Thu, Jul 14, 2016 at 6:04 PM, Mich Talebzadeh wrote: > Hi Gopal, > > If I recall you were working on a UI support for Hive. Currently the one > available is the standard Hadoop one on port 8088. > > Do you have a

A dedicated Web UI interface for Hive

Hi Gopal, If I recall you were working on a UI support for Hive. Currently the one available is the standard Hadoop one on port 8088. Do you have any timelines which release of Hive is going to have this facility? Thanks, Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?

Re: RegexSerDe with Filters

Thanks Dudu. Couple of additional questions. 1) I need to have the partition column in my view. I am able to add it to my view. But, will it add any performance overhead? 2) I have six 2GB files under date partition and currently I have only one partition. I always run into 'OutOfMemoryError:

Re: can't start up hive 2.1 hiveserver2/metastore services

we have multiple hive versions installed on the same machine and we use export HIVE_HOME via CLI which I think it is why we run into that issue. As suggested, we use the complete execution CLI path and it looks good. Thanks and closing this issue. regards, Q On Thu, Jul 14, 2016 at 12:27 PM, Mic

RE: Any way in hive to have functionality like SQL Server collation on Case sensitivity

Yes, java_method is a Synonym for reflect as of Hive 0.9.0 The use-case was presented by Mahender at the bottom of this thread (the emphasis is mine): “We would like to have feature in Hive where string comparison should ignore case sensitivity

Re: Any way in hive to have functionality like SQL Server collation on Case sensitivity

I think both are the same. can you elaborate a little bit more on your use case, eg a query you currently do and what the exact issue is > On 14 Jul 2016, at 09:36, Markovitz, Dudu wrote: > > Are you referring to ‘java_method‘ (or ‘reflect’)? > > e.g. > > hive> select java_method ('java.la

RE: Any way in hive to have functionality like SQL Server collation on Case sensitivity

Are you referring to ‘java_method‘ (or ‘reflect’)? e.g. hive> select java_method ('java.lang.Math','min',45,9) ; 9 I’m not sure how it serves out purpose. Dudu From: Jörn Franke [mailto:jornfra...@gmail.com] Sent: Thursday, July 14, 2016 8:55 AM To: user@hive.apache.org Subject: Re: Any way

答复: 答复: 答复: 答复: Using Spark on Hive with Hive also using Spark as its execution engine

I think you know what I said is relate to execution plan, right? My spark version is 1.5.X Regards Wenli 发件人: Mich Talebzadeh [mailto:mich.talebza...@gmail.com] 发送时间: 2016年7月14日 15:17 收件人: user 主题: Re: 答复: 答复: 答复: Using Spark on Hive with Hive also using Spark as its execution engine fine which

Re: 答复: 答复: 答复: Using Spark on Hive with Hive also using Spark as its execution engine

fine which version of spark are using for Hive execution/query engine please? Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehm

答复: 答复: 答复: Using Spark on Hive with Hive also using Spark as its execution engine

I using 1.x latest,but ,I checked the master branch(2.x), the latest code, no update. Regards Wenli 发件人: Mich Talebzadeh [mailto:mich.talebza...@gmail.com] 发送时间: 2016年7月14日 15:02 收件人: user 主题: Re: 答复: 答复: Using Spark on Hive with Hive also using Spark as its execution engine Wjich v

Re: 答复: 答复: Using Spark on Hive with Hive also using Spark as its execution engine

Wjich version of Hive and Spark please? Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com *Disclaimer:* Use