Signed Jars in Distributed Cache

Hi, I am trying to use a signed jar (BouncyCastle) in one of my UDFs. Since signed jars shouldn't be extracted I am using two ADD JAR statements to add both the UDF jar as well as BouncyCastle provider jar to the classpath. However, hive/hadoop automatically extracts the signed jar (BouncyCastle)

Re: Re: ERROR OutOfMemoryError: Java heap space

Use yarn logs -applicationId to see detail log. r7raul1...@163.com From: Jadhav Shweta Date: 2015-02-26 15:00 To: user Subject: Re: ERROR OutOfMemoryError: Java heap space Hi, I tried running same query from Simple JDBC Client instead of Spring Batch framework its was working fine. But now

Re: ERROR OutOfMemoryError: Java heap space

Hi, I tried running same query from Simple JDBC Client instead of Spring Batch framework its was working fine. But now when I am running it again I am getting new error Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0 2015-02-26 11:41:50,233 Stage-1 map = 0%,  red

Re: Reading tables will lock entire database in Hive 0.13

What lock manager are you using? Alan. Jim Green February 25, 2015 at 17:24 Hi Team, Here is old thread about this issue: http://qnalist.com/questions/5091386/doubt-about-locking-mechanism-in-hive I met the same issue that when reading one table in session A, we

Reading tables will lock entire database in Hive 0.13

Hi Team, Here is old thread about this issue: http://qnalist.com/questions/5091386/doubt-about-locking-mechanism-in-hive I met the same issue that when reading one table in session A, we can not create a table in the same database in session B. Basically it causes database level lock in Hive 0.13

Re: [ANNOUNCE] New Hive PMC Member - Sergey Shelukhin

Thanks guys! On 15/2/25, 16:02, "Xiaobing Zhou" wrote: >Congrats Sergey! > >On Feb 25, 2015, at 1:56 PM, Prasanth Jayachandran > wrote: > >> Congrats Sergey! >> >> On Feb 25, 2015, at 1:50 PM, Alexander Pivovarov >>mailto:apivova...@gmail.com>> wrote: >> >> Congrats! >> >> On Wed, Feb 25, 201

Re: [ANNOUNCE] New Hive PMC Member - Sergey Shelukhin

Congrats Sergey! On Feb 25, 2015, at 1:50 PM, Alexander Pivovarov mailto:apivova...@gmail.com>> wrote: Congrats! On Wed, Feb 25, 2015 at 12:33 PM, Vaibhav Gumashta mailto:vgumas...@hortonworks.com>> wrote: Congrats Sergey! On 2/25/15, 9:06 AM, "Vikram Dixit" mailto:vik...@hortonworks.com>> w

Re: [ANNOUNCE] New Hive PMC Member - Sergey Shelukhin

Congrats! On Wed, Feb 25, 2015 at 12:33 PM, Vaibhav Gumashta < vgumas...@hortonworks.com> wrote: > Congrats Sergey! > > On 2/25/15, 9:06 AM, "Vikram Dixit" wrote: > > >Congrats Sergey! > > > >On 2/25/15, 8:43 AM, "Carl Steinbach" wrote: > > > >>I am pleased to announce that Sergey Shelukhin has

Re: ERROR OutOfMemoryError: Java heap space

Hi, auth=nosasl should be only provided in JDBC url if "hive.server2.authentication” is explicitly set to NOSASL in your hive-site. Otherwise, the jdbc client tries to open a plain socket while the the server is expecting a sasl negotiation to happen. Due to an issue with thrift, this results

Re: GSS initiate failed exception

Looks like your Kerberos authentication failed. Did you renew the ticket? Typical expiry is 24 hours. From: , Chandra Reddy mailto:chandra.bog...@gs.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Wednesday, February 25, 2015 at 3:42 AM To

Re: [ANNOUNCE] New Hive PMC Member - Sergey Shelukhin

Congrats Sergey! On 2/25/15, 9:06 AM, "Vikram Dixit" wrote: >Congrats Sergey! > >On 2/25/15, 8:43 AM, "Carl Steinbach" wrote: > >>I am pleased to announce that Sergey Shelukhin has been elected to the >>Hive >>Project Management Committee. Please join me in congratulating Sergey! >> >>Thanks. >

get max partition column value

Hi Everyone Lets say I have a table partitioned by period string how to select max period? if I run select max(period) from invoice; hive 0.13.1 runs MR which is slow OK STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Tez Edges:

Re: CombineHiveInputFormat does not call getSplits on custom InputFormat

Hi, There¹s a special interface in hive-1.0, which gives more information to the input format. https://hive.apache.org/javadocs/r1.0.0/api/ql/org/apache/hadoop/hive/ql/io /CombineHiveInputFormat.AvoidSplitCombination.html But entirely skipping combination results in so many performance problems

Re: [ANNOUNCE] New Hive PMC Member - Sergey Shelukhin

Congrats Sergey! On 2/25/15, 8:43 AM, "Carl Steinbach" wrote: >I am pleased to announce that Sergey Shelukhin has been elected to the >Hive >Project Management Committee. Please join me in congratulating Sergey! > >Thanks. > >- Carl

[ANNOUNCE] New Hive PMC Member - Sergey Shelukhin

I am pleased to announce that Sergey Shelukhin has been elected to the Hive Project Management Committee. Please join me in congratulating Sergey! Thanks. - Carl

Re: config recommendations to boost performance

Hi Rajesh, thanks for your quick response. After quitting the job, no further containers are being launched. Unfortunately I have no execution plan (EXPLAIN output) to dive into that execution in detail. Do you have recommendations of Tez/Hive parameters that influence the execution of TBs of dat

GSS initiate failed exception

Hi, My Hive jdbc client queries ( hiveserver2) to secured cluster fails with below exception after one or two days running fine from tomcat. Any idea what might be the issue? Is it a known issue? 2015-02-25 04:49:43,174 ERROR [org.apache.thrift.transport.TSaslTransport] SASL negotiation fai

Re: ERROR OutOfMemoryError: Java heap space

Is your problem solved? *Thanks & Regards,* *Srinivas T* On Wed, Feb 25, 2015 at 4:17 PM, Srinivas Thunga wrote: > Hi, > > You can set fetch size based certain datatype like blob, image, etc > > statement.setFetchSize(1000); > > Try this. > > > *Thanks & Regards,* > > *Srinivas T* > > On Wed,

Re: ERROR OutOfMemoryError: Java heap space

Hi, You can set fetch size based certain datatype like blob, image, etc statement.setFetchSize(1000); Try this. *Thanks & Regards,* *Srinivas T* On Wed, Feb 25, 2015 at 4:10 PM, Jadhav Shweta wrote: > Query is working fine if i have small data set in same table. > But its throwing error fo

Re: config recommendations to boost performance

>> A query like "select name,count(id) from table where date='2015-01-01' or date='2015-01-02' group by (name)" takes almost forever and needs to be cancelled after ~30min. >> It should have ideally scanned only the 2 partitions. Do you see any container launches after which you had to kill the jo

Re: ERROR OutOfMemoryError: Java heap space

Query is working fine if i have small data set in same table. But its throwing error for large data set Thanks Shweta Jadhav -Srinivas Thunga wrote: - To: "user@hive.apache.org" From: Srinivas Thunga Date: 02/25/2015 03:45PM Subject: Re: ERROR OutOfMemoryError: Java heap space not r

Re: ERROR OutOfMemoryError: Java heap space

not required. Make sure there will not be unnecessary looping in your code *Thanks & Regards,* *Srinivas T* On Wed, Feb 25, 2015 at 2:58 PM, Jadhav Shweta wrote: > Do I need to increase HADOOP_HEAPSIZE > I haven't set it. > I have cluster with 4 machines each having 8GB ram. > I read somewhere

Re: ERROR OutOfMemoryError: Java heap space

Do I need to increase HADOOP_HEAPSIZE I haven't set it. I have cluster with 4 machines each having 8GB ram. I read somewhere that we should mention auth=noSasl in hive jdbc url. Is it necessary Thanks Shweta Jadhav -Jadhav Shweta wrote: - To: user@hive.apache.org From: Jadhav Shweta

Re: ERROR OutOfMemoryError: Java heap space

I am running the query using spring batch framework. Thanks Shweta Jadhav -Srinivas Thunga wrote: - To: "user@hive.apache.org" From: Srinivas Thunga Date: 02/25/2015 02:43PM Subject: Re: ERROR OutOfMemoryError: Java heap space Hi, Might be problem in your code. Generally it happens

Re: Re: ERROR OutOfMemoryError: Java heap space

Try select * from table limit 1; r7raul1...@163.com From: Jadhav Shweta Date: 2015-02-25 17:38 To: user Subject: Re: ERROR OutOfMemoryError: Java heap space Hi I have installed apache hive 0.13.0. I have configured metastore as postgres db. Query is working fine in beeline command line interf

Re: ERROR OutOfMemoryError: Java heap space

Hi, Might be problem in your code. Generally it happens when there are unnecessary looping. Please check your java code. *Thanks & Regards,* *Srinivas T* On Wed, Feb 25, 2015 at 2:38 PM, Jadhav Shweta wrote: > Hi > > I have installed apache hive 0.13.0. > I have configured metastore as post

Re: ERROR OutOfMemoryError: Java heap space

Hi I have installed apache hive 0.13.0. I have configured metastore as postgres db. Query is working fine in beeline command line interface but giving Heap space error while executing using JDBC java client. Thanks Shweta Jadhav -Srinivas Thunga wrote: - To: "user@hive.apache.org" Fro

Re: ERROR OutOfMemoryError: Java heap space

Hi, Let me know how you configured Hive? R u using cloudera? If you are installed Hive separately then have you configured Metastore? *Thanks & Regards,* *Srinivas T* On Wed, Feb 25, 2015 at 2:27 PM, Jadhav Shweta wrote: > Hi > > I am running simple select query > > select * from table; >

Re: ERROR OutOfMemoryError: Java heap space

Hi I am running simple select query select * from table; thanks Shweta Jadhav -Srinivas Thunga wrote: - To: "user@hive.apache.org" From: Srinivas Thunga Date: 02/25/2015 01:09PM Subject: Re: Hi,  Can  you place the query as well which you are trying? Thanks & Regards, Sriniva

config recommendations to boost performance

Hi, I'm a bit stuck in optimizing the hive/tez config parameters to speed up Hive/Tez query execution. The cluster consists of 6 worker nodes (with rather hadoop-non-ideal component proportion, but that's given) including: 48Cores/384GB Ram/10HDDs. The Hive table is configured as: - partitioned by