Re: Sampling from a single column

2014-02-12 Thread Navis류승우
If it should be sampled using subquery would be inevitable, something like, select x from (select distinct key as x from src)a where rand() > 0.9 limit 10; 2014-02-12 6:07 GMT+09:00 Oliver Keyes : > Hey all > > So, what I'm looking to do is get N randomly-sampled distinct values from > a colum

Re: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/HBaseConfiguration

2014-02-12 Thread Josh Draner
Hi, > Is your huve able to connect to hadoop? It appears to: hadoop$ hdfs dfs -ls /user/hive Found 1 items drwxrwxr-x - hadoop supergroup 0 2014-02-12 17:58 /user/hive/warehouse hadoop$ hive hive> CREATE TABLE myTable(id INT, name STRING); OK Time taken: 5.893 seconds hive> quit; had

Re: Compiling Hive 0.12.0

2014-02-12 Thread Prasanth Jayachandran
Bryan, This might be a bug. Are you testing with sample data? Will it be possible to send me the values in col_b? Alternatively, you can file a bug and upload the data for failing column. I can take a look at it. There is another bug that was opened recently. https://issues.apache.org/jira/b

Re: Issue with Hive and table with lots of column

2014-02-12 Thread Navis류승우
With HIVE-3746, which will be included in hive-0.13, HiveServer2 takes less memory than before. Could you try it with the version in trunk? 2014-02-13 10:49 GMT+09:00 Stephen Sprague : > question to the original poster. closure appreciated! > > > On Fri, Jan 31, 2014 at 12:22 PM, Stephen Sprag

Re: Issue with Hive and table with lots of column

2014-02-12 Thread Stephen Sprague
question to the original poster. closure appreciated! On Fri, Jan 31, 2014 at 12:22 PM, Stephen Sprague wrote: > thanks Ed. And on a separate tact lets look at Hiveserver2. > > > @OP> > > *I've tried to look around on how i can change the thrift heap size but > haven't found anything.* > > > lo

Re: Compiling Hive 0.12.0

2014-02-12 Thread Bryan Jeffrey
Prasanth, I applied the patched Hive and restarted Hadoop/Hive. I selected from a text table into a newly created ORC table, and found the same errors. Perhaps this is a different problem? hive> select min(col_b), max(col_b), count(col_b) from text_data_range_572643; 46 411592614 572

Re: Compiling Hive 0.12.0

2014-02-12 Thread Prasanth Jayachandran
Hi Bryan HIVE-5991 is a writer bug. From the exception I see that exception happens while reading the ORC file. It might happen that its trying to read a corrupted ORC file. Are you trying to read the old ORC file after applying this patch? Since it is a writer bug, the ORC file needs to be reg

Compiling Hive 0.12.0

2014-02-12 Thread Bryan Jeffrey
Hello. I am running Hive 0.12.0 & Hadoop 2.2.0. I attempted to apply the fix described in the patch here: https://issues.apache.org/jira/secure/attachment/12617931/HIVE-5991.1.patch I applied the patch, and ran 'ant tar' from the src directory. A tar file for distribution was created, and it ap

Re: Can data be passed to the final mode init call in a UDAF?

2014-02-12 Thread Jason Dere
I've only used Hive 0.12+ and I'm not really familiar with UDAFs, but had to pass some info to a GenericUDF so this might give you some ideas: - During query planning you might be able to set the hive conf using SessionState.get().getConf(). When the query is executing you might be able to acc

Re: Can data be passed to the final mode init call in a UDAF?

2014-02-12 Thread John Meagher
The code I'm having problems with is https://github.com/livingsocial/HiveSwarm/blob/WIP_scripting/src/main/java/com/livingsocial/hive/udf/ScriptedUDAF.java. There is a FIXME comment at the problem area where the return type for the init method can not be determined. On Mon, Feb 10, 2014 at 4:26 P

0.13.0 Hive Issue - Hive client connecting via thrift

2014-02-12 Thread Bryan Jeffrey
Hello. We are running Hadoop 2.2.0, Hive 0.12.0. We're seeing an error reading integer fields from ORC tables that looks very similar to HIVE-5991. In an attempt to fix the issue, I downloaded Hive 0.13.0 trunk and compiled it. On starting Hive 0.13.0, I see the following messages: 2014-02-12

Reading hive table using Pig script

2014-02-12 Thread Chhaya Vishwakarma
Hi, I am trying to read hive table using PIG script but when I run a pig code to read a table in hive its giving me following error 2014-02-12 15:48:36,143 [main] WARN org.apache.hadoop.hive.conf.HiveConf - hive-site.xml not found on CLASSPATH 2014-02-12 15:49:10,781 [main] ERROR org.apache.p