Re: Issue while using hive 0.9

No I don't see any antlr jars in the output. Infact not even the default version that exists in the hive lib folder. Do I need to explicitly mention in the classpath? On Tuesday 18 September 2012 11:16 AM, Ashutosh Chauhan wrote: Do ps aux | grep hive n check the classpath of hive process. See

Re: Issue while using hive 0.9

Do ps aux | grep hive n check the classpath of hive process. See if you can spot antlr jar versions in it. On Mon, Sep 17, 2012 at 10:40 PM, Sarath < sarathchandra.jos...@algofusiontech.com> wrote: > Thanks for the quick reply. Tried it. But same issue. > > Pointed system to JDK 1.6 (update 32).

Re: Issue while using hive 0.9

Thanks for the quick reply. Tried it. But same issue. Pointed system to JDK 1.6 (update 32). Changed hadoop conf to use this JDK. "java -version" prints the 1.6 version. Replaced antlr JAR with the same version as it came in hive binary (3.0.1). Anything else I have to do? Do I need to make any

Re: Issue while using hive 0.9

Only thing which is different then usual setups is jdk1.7 Is it possible for you to test with jdk1.6 ? Ashutosh On Mon, Sep 17, 2012 at 10:02 PM, Sarath Chandra < sarathchandra.jos...@algofusiontech.com> wrote: > @Ashutosh > I haven't built my hive from sources. I just downloaded the binaries fro

Re: Issue while using hive 0.9

@Ashutosh I haven't built my hive from sources. I just downloaded the binaries from the site. The downloaded tar has come with 3.0.1 version of antlr. I didn't do any changes. Just set the environment variables and invoked hive and got this error. I checked out sources and tried to build hive as di

Re: Questions about Hive

Yes, Hive is meant for batch processing on a very large data set. It's very latent when compared to other "databases" such as, MySQL, but excels where other databases faulter. For example, running analysis on several terabytes of data is not unusual in Hive. It was mentioned to consider HBase,

Re: Issue while using hive 0.9

Hi Sarath, Sorry for delay – I don't see this mail folder all the time. Anyhow do try with exactly antler 3.4.1 – available at http://www.antlr.org/wiki/display/ANTLR3/ANTLR+3+Wiki+Home Even antler 3.0.1 might work. Thanks Rekha From: Sarath mailto:sarathchandra.jos...@algofusiontech.com>> R

Incomplete example on page cwiki.apache.org/Hive/compressedstorage.html

Hey guys, I spent a lot of time to figure out that when I use the example from the page cwiki.apache.org/Hive/compressedstorage.html: SET hive.exec.compress.output=true; SET io.seqfile.compression.type=BLOCK; -- NONE/RECORD/BLOCK (see below) INSERT OVERWRITE TABLE raw_sequence SELECT * FROM raw;

Re: Questions about Hive

I believe Hive is not for web users, since it takes several minutes or even hours to do one query. But I managed to provide a web service via THRIFT and php. http://nousefor.net/55/2011/12/php/hbase-and-hive-thrift-php-client/ On Mon, Sep 17, 2012 at 10:42 PM, Hamilton, Robert (Austin) wrote: > He

Re: hive json serde

Thanks Chuck. I tried both json_tuple & get_json_objects on top of the serde table and it works too. I have nested jsons and some of the keys do not have a specific schema (You can throw whatever you want there) so if I want to query from those keys then I have to use get_json_object on top of the

RE: hive json serde

A few things that should help... - There are three ways to handle JSON within a Hive table, get_json_object, json_tuple, and JSON SerDe. These are separate mechanisms. So if you want to use get_json_object, you don't need to load a SerDe. - With a SerDe, Hive understands JSON fields natively, s

Re: hive json serde

It works now. Looks like there is a bug in the code. if you do hive --auxpath ./serde then I get an error but if I get the full path as hive --auxpath /var/lib/hdfs/serde/ then get_json_object() works. Thanks for your help. Appreciate it. CCing hive user group as well. On Mon, Sep 17, 2012 at

Re: How to overwrite a file inside hive table folder

Hi, I think you could try to drop partition and create a new partition if you table is partitioned Ruslan On Mon, Sep 17, 2012 at 8:12 AM, MiaoMiao wrote: > What do you mean by "a file"? If your HDFS dir contains several file > and you want to overwrite one, then no way you can do it with HiveQ

Re: date format question

You could use the unix_timestamp function ... unix_timestamp(ts, ) >= unix_timestamp("2012-09-10", '-MM-dd') ... something on those lines Also checkout https://cwiki.apache.org/Hive/languagemanual-udf.html#LanguageManualUDF-DateFunctions for more datetime functions in Hive. Ashish On Mon,

Re: hive json serde

Thanks Chuck. I'll try this now. On Mon, Sep 17, 2012 at 12:10 PM, Connell, Chuck wrote: > I used his pre-built jar. No need to compile anything. > > ** ** > > Be sure to add “ --auxpath /path/to/jar/dir “ to the Hive command line.*** > * > > ** ** > > Chuck > > ** ** > > ** ** > > ** **

date format question

I have a date column in this format: Mon Jul 16 14:05:12 PDT 2012 I want to select all records > 2012-09-01. How to do? Thanks!

RE: hive json serde

I used his pre-built jar. No need to compile anything. Be sure to add " --auxpath /path/to/jar/dir " to the Hive command line. Chuck From: Connell, Chuck [mailto:chuck.conn...@nuance.com] Sent: Monday, September 17, 2012 3:06 PM To: user@hive.apache.org Subject: RE: hive json serde I just fin

RE: hive json serde

I just finished testing this one. No problems found. The developer is also quite responsive to issues raised. I encouraged him to submit it to the Hive dev team as core code. https://github.com/rcongiu/Hive-JSON-Serde/ Chuck Connell Nuance R&D Data Team Burlington, MA From: Mark Golden [mail

hive json serde

Hi, I'm looking for a hive json serde that works well. Any suggestions? I did a quick google and found a few but none of them are complete. Thanks, Mark.

Re: Issue while using hive 0.9

That won't work. You must use the exact same version which was used while compiling Hive, i.e., 3.0.1 any other version won't work. Look at the classpath of java process of Hive to make sure there is no other version in your classpath. Hope it helps, Ashutosh On Mon, Sep 17, 2012 at 4:40 AM, Sara

RE: Questions about Hive

Hello, something :) Regarding jdbc style: I understand this approach has some limitations, but here is an example. You will need to make sure the hive service is running: https://cwiki.apache.org/Hive/hiveserver.html Here is a sample code that I've used for testing. It is not the best java in th

Re: warning message while connecting Hive shell

Same for me. It's been there always, didn't really bother, however if there's a fix, we should fix it. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for

Re: Problem importing data

Hi Manish, Thank you for your response. I can't really see why this would happen as we used the same format in an other cluster that worked well. Is this a version problem? We do actually have a date field in the rows. Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverla

Segfault in Python script during Transform

I am running a transform query in hive 0.9.0 and trying to figure out why my script segfaults while run as part of a hive job but not part when run by itself. The library that is freaking out is libcrypto and is being called by the Python m2crypto module. That being said, I can run the same pytho

Re: Show job progress when using JDBC to run HIVE query

Thanks a lot for all the answers and suggestions. Looks like one hacky workaround is to check the hadoop task status. But for my project it's way too much cost. On Mon, Sep 17, 2012 at 8:31 AM, Bennie Schut wrote: > The jdbc driver uses thrift so if thrift can't then jdbc can't. > > This can be

RE: warning message while connecting Hive shell

I get the same warning. Could you please be more specific about "set the class path wrt log4J property file"? Exactly what should we do? Chuck Connell Nuance R&D Data Team Burlington, MA -Original Message- From: ashok.sa...@wipro.com [mailto:ashok.sa...@wipro.com] Sent: Monday, Septemb

RE: Show job progress when using JDBC to run HIVE query

The jdbc driver uses thrift so if thrift can't then jdbc can't. This can be surprisingly difficult to do. Hive can split a query into x hadoop jobs and some will run in parallel and some will run in sequence. I've used oracle in the past (10 and 11) and I could also never find out how long a lar

RE: warning message while connecting Hive shell

Please set the class path wrt log4J property file Thanks Ashok S. From: Ajit Kumar Shreevastava [ajit.shreevast...@hcl.com] Sent: Monday, September 17, 2012 4:11 PM To: user@hive.apache.org Subject: warning message while connecting Hive shell Hello , When

Re: Issue while using hive 0.9

From ANTLR site downloaded the latest JAR -> antlr-3.4-complete.jar. Copied this to lib folder and tried again. Same issue persists. Thanks, Sarath. On Monday 17 September 2012 04:42 PM, Joshi, Rekha wrote: Try with antlr 3.4.1 jar? Thanks Rekha From: Sarath

Re: Issue while using hive 0.9

Try with antlr 3.4.1 jar? Thanks Rekha From: Sarath mailto:sarathchandra.jos...@algofusiontech.com>> Reply-To: mailto:user@hive.apache.org>> Date: Mon, 17 Sep 2012 16:24:02 +0530 To: mailto:user@hive.apache.org>> Subject: Issue while using hive 0.9 Hi, I'm new to hive. As first step downloade

Issue while using hive 0.9

Hi, I'm new to hive. As first step downloaded version 0.9 and upacked it. Set the HIVE_HOME to the unpacked folder. I already have hadoop 1.0.3 installed and its configuration is in classpath. Then issued the hive command and got the hive prompt. After this whatever I try to execute, results

RE: Problem importing data

Robin, I think this is type casting issue with date column. Can you send the table definition and sample data then it would be easy to figure out where it is giving datatype issue? My suggestion is to create a staging table first where you dump the data considering all the column as STRING and

warning message while connecting Hive shell

Hello , When I want to make connection with hive I got the following warning message. [hadoop@NHCLT-PC44-2 conf]$ hive WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files. Logging initialized u

Problem importing data

Hi there, We're trying to load in lots of data into Hive, however the import command fails: *WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.* *Logging initialized using configuration in jar

Re: Does hive support nested queries?

Yes, + is an operator. t1 a b 1 2 3 4 and the result will be: col 3 7 On Mon, Sep 17, 2012 at 4:43 PM, Amila Maha Arachchi wrote: > Hi Bharath, > > Thanks for the reply. But I am confused while trying to understand the > example in the wiki. Example says > > > SELECT col > FROM ( > SELECT a+b

Re: Does hive support nested queries?

Hi Bharath, Thanks for the reply. But I am confused while trying to understand the example in the wiki. Example says SELECT col FROM ( SELECT a+b AS col FROM t1 ) t2 Can you please explain me what is a+b here. Are they two columns from table t1? Can you also tell me whether there is anot

Re: Does hive support nested queries?

Hive supports subqueries only in the FROM clause as of now. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+SubQueries On Mon, Sep 17, 2012 at 1:03 PM, Amila Maha Arachchi wrote: > Hi folks, > > I want to provide the output of a select query to a where clause of > another query a

Re: Questions about Hive

I don't think Hive is intended for web request scoped operations... that would be a rather unusual case from my understanding. HBase sounds more like the Hadoop equivalent that you might be looking for, but you need to look at your search patterns to see if HBase is a good fit (you need to manage