Re: Generate Hive DDL

2013-02-14 Thread Anandha L Ranganathan
1) download the jar from from Apache JIRA https://issues.apache.org/jira/browse/HIVE-967 2) run the following command to generate DDLs. hive --service jar <> com.media6.hive2rdbms.job.ShowCreateTable -D db.name=default -D table.name=<> 2>/dev/null On Thu, Feb 14, 2013 at 9:41 PM, Mur

Generate Hive DDL

2013-02-14 Thread Murtaza Doctor
Folks, I was wondering if any one is aware of a way to generate Hive DDL scripts using the existing metastore which in our case is in postgres. We have a few tables which have been historically created and figuring our creating ways to get the schema as DDL so that we can keep it in under versi

Re: Map join optimization issue

2013-02-14 Thread Aniket Mokashi
I think hive.mapjoin.smalltable.filesize parameter will be disregarded in that case. On Thu, Feb 14, 2013 at 7:25 AM, Mayuresh Kunjir wrote: > Yes, the hint was specified. > On Feb 14, 2013 3:11 AM, "Aniket Mokashi" wrote: > >> have you specified map-join hint in your query? >> >> >> On Thu, Fe

Fwd: Help to solve UDAF errors!

2013-02-14 Thread Abhishek Bhattacharya
Hello, This is in reference to the below mentioned UDAF. As you can notice, I am collecting all the elements in a list for each mapper. Now, at the reducer I merge all the lists and calculate the top N percent records to return back from the terminate(). Now, clearly this is not a scalable solutio

Hive ServerSocket

2013-02-14 Thread Cyrille Djoko
Hi, intern here. My setup is the following, Hadoop 1.0.4, Sqoop 1.4.2, Hive 0.9.0 Hardware: Memory 495.6 MiB Processor Intel(R) Pentium(R) 4 CPU 2.66GHz Ubuntu 10.04 Lucid I wrote a small program that would connect to the local hive, grabs data from a given table and then crea

Hive ServerSocket

2013-02-14 Thread cd
Hi, intern here. My setup is the following, Hadoop 1.0.4, Sqoop 1.4.2, Hive 0.9.0 Hardware: Memory 495.6 MiB Processor Intel(R) Pentium(R) 4 CPU 2.66GHz Ubuntu 10.04 Lucid I wrote a small program that would connect to the local hive, grabs data from a given table and then crea

Re: HBase-Hive integration - fetching long values in HBase as bigint in Hive

2013-02-14 Thread Ashish Nigam
It worked... I was missing this in SERDE properties - "hbase.table.default.storage.type" = "binary" On Thu, Feb 14, 2013 at 9:02 AM, Ashish Nigam wrote: > Columns in "stats-daily" CF are dynamic in nature but all the values are > long counters. > I also tried with a table that has only one co

Re: HBase-Hive integration - fetching long values in HBase as bigint in Hive

2013-02-14 Thread Ashish Nigam
Columns in "stats-daily" CF are dynamic in nature but all the values are long counters. I also tried with a table that has only one column mapped. That also is returning NULL. So the issue does not seem to be different value types in a column family. On Thu, Feb 14, 2013 at 6:51 AM, kulkarni.swa

Re: Nullpointer Exception when using UDF

2013-02-14 Thread Nitin Pawar
Neelesh, Will it be possible for you to share your code? Looks like in your UDF you are not handling NULL as input values. Thanks, Nitin On Thu, Feb 14, 2013 at 10:22 PM, neelesh gadhia wrote: > Hi Dean, > > Thanks for your response. I reviewed the stack trace. As you mentioned > the error

Re: Nullpointer Exception when using UDF

2013-02-14 Thread neelesh gadhia
Hi Dean, Thanks for your response.  I reviewed the stack trace. As you mentioned the error shows up at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:137) But this probably is  a java class that comes with hadoop 1.1.1 and it untouched.

Re: CHAN (Comprehensive Hive Archive Network) (Was Re: Using Reflect: A thread for ideas)

2013-02-14 Thread Patrick D'Souza
I was having a similar discussion with some of our Hadoop Admins at About.com last week about a central repository for UD[A,T]Fs. Most often we're trying to accomplish the same thing with UD[A,T]Fs so a central repository can make live easy for everyone. On Thu, Feb 14, 2013 at 8:58 AM, Connell,

Re: Map join optimization issue

2013-02-14 Thread Mayuresh Kunjir
Yes, the hint was specified. On Feb 14, 2013 3:11 AM, "Aniket Mokashi" wrote: > have you specified map-join hint in your query? > > > On Thu, Feb 7, 2013 at 11:39 AM, Mayuresh Kunjir < > mayuresh.kun...@gmail.com> wrote: > >> >> Hello all, >> >> >> I am trying to join two tables, the smaller bein

Re:

2013-02-14 Thread Dean Wampler
According to your stack trace, you have NullPointerException on line 137 of your UDF. On Thu, Feb 14, 2013 at 2:28 AM, neelesh gadhia wrote: > Hello, > > I am a Newbie to using UDF's on hive. But implemented these GenericJDF ( > https://issues.apache.org/jira/browse/HIVE-2361 ) on hive 0.9.0 and

RE: CHAN (Comprehensive Hive Archive Network) (Was Re: Using Reflect: A thread for ideas)

2013-02-14 Thread Connell, Chuck
+1, great idea! Chuck Connell, Nuance From: Robin Morris [mailto:r...@baynote.com] Sent: Thursday, February 14, 2013 1:59 AM To: user@hive.apache.org Subject: CHAN (Comprehensive Hive Archive Network) (Was Re: Using Reflect: A thread for ideas) I think we need to think a little bigger than this

Re: CHAN (Comprehensive Hive Archive Network) (Was Re: Using Reflect: A thread for ideas)

2013-02-14 Thread John Omernik
I think the idea has potential. It would be cool if there was a sort of (excuse the analogy) social media like system that would allow community to identify which UD[AT]Fs or reflect type functions are useful to them. It would almost act as a staging platform to help identify to the maintainers

[no subject]

2013-02-14 Thread neelesh gadhia
Hello, I am a Newbie to using UDF's on hive. But implemented these GenericJDF ( https://issues.apache.org/jira/browse/HIVE-2361 ) on hive 0.9.0 and hadoop 1.1.1. Was able to add jar to hive hive> select * from emp; OK 1    10    1000 2    10    1200 3    12    1500 4    12    300 5    12    18

Re: Map join optimization issue

2013-02-14 Thread Aniket Mokashi
have you specified map-join hint in your query? On Thu, Feb 7, 2013 at 11:39 AM, Mayuresh Kunjir wrote: > > Hello all, > > > I am trying to join two tables, the smaller being of size 4GB. When I set > hive.mapjoin.smalltable.filesize parameter above 500MB, Hive tries to > perform a local task to

Re: Lateral View in sub query issue

2013-02-14 Thread Aniket Mokashi
Most likely its : HIVE-3226 (HIVE-1901) Workaround: set hive.optimize.cp=false On Sat, Jan 26, 2013 at 10:40 AM, Mark Grover wrote: > Hi John, > Thanks for reporting this. > > Can you please take a look at the Lateral View issues here: > > https://issues.apache.org/jira/issues/?jql=project%20%3D