Re: SerDe errors

-- Good judgement comes with experience. Experience comes with bad judgement. -- Roberto Congiu - Data Engineer - OpenX tel: +1 626 466 1141

Re: how to load json with nested array into hive?

ma to automate Hive schema generation from >> JSON: >> https://github.com/quux00/hive-json-schema >> >> >> From: Nitin Pawar mailto:nitinpawar...@gmail.com >> >> >> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" < >>

Re: Simple UDF to return array

p as a > parameter > > I tried cast to array and that fails as well. > > Requesting help from the community. I am considering writing generic UDF, > but this is a simple requirement and would like to be able to use simple > UDF if I can. > > regards > Sunita > > &

Re: JSON data to HIVE table

what is the method I need to follow? >> >> Thanks, >> Raj >> > > > > -- > Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome. > com > -- -- Good judgement comes with experience. E

Re: HiveServer2

d that turn it off. Other then that we just > wrote a good nagios check..it runs a query (one that does not invoke a map > reduce job). That seems to spot the problems quickly and allow our ops to > restart the bad instance. > > > On Mon, Nov 18, 2013 at 5:11 PM, Roberto Congiu

Re: HiveServer2

ting+up+HiveServer2 >> >> Anyhoo. Consider this a Public Service Announcement. Take heed. >> >> Regards, >> Stephen. >> >> >> >> > -- -- Good judgement comes with experience. Experience

Re: SerDe for Fixed Columns?

received this >> communication in error, please notify the sender and delete all copies of >> this message. Persistent Systems Ltd. does not accept any liability for >> virus infected mails. >> > > > > -- > Nitin Pawar > -- -- Good judgement comes with experience. Experience comes with bad judgement. -- Roberto Congiu - Data Engineer - OpenX tel: +1 626 466 1141

IMPORTANT: access to ox3 non-archived hive tables

ltogether, except for mkt_archived_hourly. R. -- -- Good judgement comes with experience. Experience comes with bad judgement. ---------- Roberto Congiu - Data Engineer - OpenX tel: +1 626 466 1141

Generic UDFs and named parameters

udgement. ------ Roberto Congiu - Data Engineer - OpenX tel: +1 626 466 1141

Re: BytesWritable

h any attachments, contains > information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station, > New Jersey, USA 08889), and/or its affiliates Direct contact information > for affiliates is available at > http://www.merck.com/contact/contacts.html) that may be confidential, > proprietary copyrighted and/or legally privileged. It is intended solely > for the use of the individual or entity named on this message. If you are > not the intended recipient, and have received this message in error, > please notify us immediately by reply e-mail and then delete it from > your system. > -- -- Good judgement comes with experience. Experience comes with bad judgement. -- Roberto Congiu - Data Engineer - OpenX tel: +1 626 466 1141

UDF for transforming a collection of arrays into an array of structs

nybody was actually interested in something like that. R. -- -- Good judgement comes with experience. Experience comes with bad judgement. ------ Roberto Congiu - Data Engineer - OpenX tel:

Re: Interference between two customSerde.jar

> run a query? > > All help will be welcome :) > > Thanks and regards, > > Germain. > > -- -- Good judgement comes with experience. Experience comes with bad judgement. -- Roberto Congiu - Data Engineer - OpenX tel: +1 626 466 1141

Re: SerDe loading external scheme

I think the additions you make to the Configuration object in the client are done after the job.xml is written to disk and sent to the rest of the cluster, but you could add the external resource URL in hive-site.xml, add a SET external.resource.url=... in the hive query or on your hiverc, or pass

Re: Accessing elements from array returned by split() function

You can also use parse_url(url, 'HOST') instead of a regular expression. On Thu, Mar 1, 2012 at 1:32 PM, Saurabh S wrote: > Of course, it works fine now. I feel like an idiot. > > And that problem using parse_url also went away and I can use that as well. > > Thanks a bunch, Phil. > > > Date: T

Re: Adding Hive Partitions Programatically.

Also consider that if you're using the metastore API, you'll most likely need to recompile whenever you upgrade hive, while you most likely won't need to do that if you use thrift. R. On Mon, Feb 27, 2012 at 9:58 AM, Edward Capriolo wrote: > You can use the metastore API directly if you source

Re: 2 questions about SerDe

Have a look at the code for the LazySerDes. When you deserialize in the SerDe, you don't actually have to deserialize all the columns. Deserialized could return an object that is not actually deserialized and you can write an ObjectInspector that deserializes a field from that structure but only wh

Re: Executing queries on rows from multiple tables

You can use UNION even if the tables are different: SELECT * FROM ( select_statement UNION ALL select_statement ) unionResult https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Union On Mon, Feb 13, 2012 at 7:10 PM, Sameer Agarwal wrote: > Hi, > > Is there a straightforward w

Re: external partitioned table

Hi Koert, we have a similar situation and this is what we did. In our case, the partitions correspond to dates. We also have multiple external tables set up this way. The upstream process updates a status file with the earliest and latest date available. I scan the DFS for new partitions (scan prog

Re: error on left/right join, hive 0.8.0

27;m not sure which version of Hive you're using. If you're talking about the > version of Hive > that comes with Cloudera's distribution then it can't be version 0.8.0 > because we > have not yet included that version in CDH. > > > On Thu, Jan 19, 2012 at 5

error on left/right join, hive 0.8.0

Hey guys, we found an issue that looks like a bug (hive 0.8 , caldera's distribution). SELECT count(1) FROM table1 a LEFT OUTER JOIN table2 b ON ( a.key1 = b.key1 AND a.key2 = b.KEY2) fails with java.lang.IllegalArgumentException: Can not create a Path from an empty string ( full stack trac

hiveserver not picking up the files in HIVE_AUX_JARS_PATH

Hey guys, I ran into a quite annoying issue. I have some jars in HIVE_AUX_JARS_PATH that include some serdes and UDF. They work fine from CLI, but when I run the same queries through hiveserver (submitting them through thrift) I get errors on the tasktrackers, even though: - I specify the HIVE_AUX

Re: nested JSON structures

I wrote a SerDe that can handle nested structures. https://github.com/rcongiu Roberto On Fri, Dec 9, 2011 at 7:42 AM, Ibrahim Acet wrote: > Hi, > > are there any tutorials available for custom UDF creation ? > > ** ** > > thx > > ** ** > > *Von:* Edward Capriolo [mailto:edlinuxg..

Re: JsonSerde usage help

I think the struct for a struct is correct (you can see in ObjectInspectorUtils.getStandardStructTypeName() how it is built). If you're using this serde: http://code.google.com/p/hive-json-serde/source/browse/trunk/src/org/apache/hadoop/hive/contrib/serde2/JsonSerde.java, it just doesn't support s

Specify a SerDe for a partition

one. R. -- Roberto Congiu -Data Engineer - OpenX 20 E Del Mar blvd, Pasadena, CA

JSON SerDe

ployer OpenX has given permission to contribute it to the community. R. -- Roberto Congiu -Data Engineer - OpenX 20 E Del Mar blvd, Pasadena, CA

Re: Can Hive 0.7 Rebuild partitions ?

PARTITION commands. >>> >>> Is there an easier way to have hive scan the base table folder to see if >>> there are any new partitions around? Something like REBUILD PARTITIONS >>> perhaps?? >>> >>> >>> >>> >>> Couldn't find anything similar on the Hive/LanguageManual/DDL >>> >>> -- >>> Kind Regards >>> >>> >>> >>> Jasper >>> >> >> > -- Roberto Congiu -Data Engineer - OpenX 20 E Del Mar blvd, Pasadena, CA