Re: Hiveserver2 memory / thread leak v 0.13.1 (hdp-2.1.5)

2015-02-06 Thread Alexander Pivovarov
Is HIVE-7353 relates to embedded MetaStore only? We use MySQL for metadata. HS2 OOM yesterday Current HS2 instance uses just 77MB for now (39 live threads). I found another issue fixes HS2 leaks https://issues.apache.org/jira/browse/HIVE-9234 JVM Arguments: -Xmx1024m -Djava.net.preferIPv4Stack=

Re: Compile hive-0.10.0 failure due to XML pasing issues

2015-02-06 Thread Zhuo Liu
A little more info: /data/xwang/hive-0.10.0/src/build.xml:538: The following error occurred while executing this line: /data/xwang/hive-0.10.0/src/builtins/build.xml:37: The following error occurred while executing this line: /data/xwang/hive-0.10.0/src/pdk/scripts/build-plugin.xml:110: javax.x

Question on support for org.hadoop.mapreduce.lib.input.FileInputFormat

2015-02-06 Thread Rui Xue
Hello all, I am new Hive researcher. Currently I intend to create a self-custom FileInputFormat for special research purpose. It seems that the current Hive (0.14.0 or higher) still support only the previous FileInputFormat i.e. org.apache.hadoop.mapred.FileInputFormat, but it would be a lot easie

Compile hive-0.10.0 failure due to XML pasing issues

2015-02-06 Thread Zhuo Liu
Dear everyone, Have you met the following issues when compiling hive-0.10.0? Thanks very much hive-0.10.0/src/$ ant clean package BUILD FAILED /data/xwang/hive-0.10.0/src/build.xml:538: The following error occurred while executing this line: /data/xwang/hive-0.10.0/src/builtins/build.xml:37: T

[ANN] Hivemall v0.3 is now available

2015-02-06 Thread Makoto Yui
Hello all, We are excited to announce that a new stable version of Hivemall (v0.3.0) is now available. https://github.com/myui/hivemall/releases/tag/v0.3.0 Hivemall provides a collection of machine learning algorithms as Hive UDFs/UDAFs/UDTFs. The main enhancement in v0.3.0 is the support f

how to get the return code of beeline in shell?

2015-02-06 Thread 李爽
my command is like this: beeline —f /tmp/x the sql script is in /tmp/x how can i get the return code of beeline to judge the result of beeline execution? thanks.

Re: Hiveserver2 memory / thread leak v 0.13.1 (hdp-2.1.5)

2015-02-06 Thread Vaibhav Gumashta
Alexander, Can you share the jmap histo (or even better, a heapdump)? What are the top consumers? What are the heap+permgen sizes that the JVM is configured to use? FYI, we fixed this memory leak in Hive-14: https://issues.apache.org/jira/browse/HIVE-7353. Thanks, -Vaibhav From: Alexander Pi

RE: COMMERCIAL:Re: Hive - regexp_replace function for multiple strings

2015-02-06 Thread Matthew Dixon
Below 2 solutions. Solution1 uses lookahead and lookbehind but works with bi-grams only. It also doesn’t enforce the pairs you’re asking for, so for instance hip music would become hipmusic. Solution2 uses simple IN syntax with if(), works with n-grams beyond bi-grams and enforces the actual