Question on converting an array of structs into a map

2014-04-08 Thread Sunderlin, Mark
I have data in two tables that looks like this: Table_1 Sequence_numint, User_id string Table_2 Sequence_numint User_attributes array> Sequence number joins the two and it is a one to one and only one relationship. Where I want to end up: A singl

Size of a Hive Map column in characters!

2013-12-06 Thread Sunderlin, Mark
The size(map) function is defined as follows: size(Map)Returns the number of elements in the map type What if I want the total size of the map for that row? This doesn't work: select length(MAP); How can I get the total size of a map column in either bytes or characters? --- Mark E. Sunder

RE: Hive Query via Hue, Only column headers in downloaded CSV or XSL results, sometimes

2013-09-30 Thread Sunderlin, Mark
| C: 540-327-6222 | AIM: MESunderlin 22000 AOL Way, Dulles, VA 20166 -Original Message----- From: Sunderlin, Mark [mailto:mark.sunder...@teamaol.com] Sent: Wednesday, September 18, 2013 2:08 PM To: user@hive.apache.org Subject: Hive Query via Hue, Only column headers in downloaded

Hive Query via Hue, Only column headers in downloaded CSV or XSL results, sometimes

2013-09-18 Thread Sunderlin, Mark
Using Hive V11, via Hue from CDH4, I can run my query, output 10 rows (limit 10) and download to a nice CSV or XSL file ... sometimes. :-( Sometimes, even when the run is error free, the download only downloads the column headers. This is true for both the CSV and XSL options. It is only ten l

RE: Issues with Hive 'Like" and multiple '%' wildcard in a pattern

2013-06-13 Thread Sunderlin, Mark
like '%West%v%'; +-+ | city    | +-+ | West Haven  | | West Haven  | | West Haven  | | West Haven  | | West Covina | | West Covina | +-----+ 6 affected hisql>select city from junk where city like '%West%v%a%'; +-+ | city    |

RE: Issues with Hive 'Like" and multiple '%' wildcard in a pattern

2013-06-12 Thread Sunderlin, Mark
: Re: Issues with Hive 'Like" and multiple '%' wildcard in a pattern that seems pretty hard to believe. what version of hive are you using? On Wed, Jun 12, 2013 at 6:27 AM, Sunderlin, Mark mailto:mark.sunder...@teamaol.com>> wrote: This seems to work just fine in other S

Issues with Hive 'Like" and multiple '%' wildcard in a pattern

2013-06-12 Thread Sunderlin, Mark
This seems to work just fine in other SQLs, but doesn't seem work in hive. I need to have several wild card characters in my 'like' clause as follows. In other SQLs, I want: where page_url_query like '%?icid=main%dl%' But in Hive that doesn't match. I have several work arounds. I can w

RE: why I get Table not found 'dual'

2012-10-18 Thread Sunderlin, Mark
meone accidentally adds a second row or deletes the first row dual stops working right. This was why I wrote dual input format. It always works right regardless if the table has data. Edward On Thu, Oct 18, 2012 at 12:09 PM, Sunderlin, Mark wrote: > I creates a single column, single row

RE: why I get Table not found 'dual'

2012-10-18 Thread Sunderlin, Mark
I creates a single column, single row table called 'dual' 1. Create a text file on HDFS, call it a.txt with just the letter 'a' (or any single character in it) 2. Create the single column table, dual: 'create table dual (x, string);' 3. Load the text file into dual: load data loca

RE: Can't use OR in left join

2012-07-26 Thread Sunderlin, Mark
How about: select    a.pid,   b.pid tab1 a   left join tab2 b   on (a.pid=b.pid); union all -- Need 'union all' to remove dups for the case of when (a.pid=b.pid) and (substr(a.pid,1,27)=b.pid) select    a.pid,   b.pid tab1 a   left join tab2 b   on (substr(a.pid,1,27)=b.pid); --- Mark E. Sunder

An array and a map in the same Hive table: Can Separator for Map KV pairs be different than Separator for Array elements?

2012-06-14 Thread Sunderlin, Mark
If my data has three columns and a typical row looks like: 5754^E ContentQuality5,Knowledge,Knowledge/Nature,UnFlagged,EarthReport^EdisplayHeight=293&displayWidth=570&imid=09177970492035608320&sid=577&skey=63&videoid=506875580 I have an integer, an array, and a map. Columns separator is a Contro

RE: How to check if all key/value pairs in two maps are equal?

2012-02-03 Thread Sunderlin, Mark
36 PM To: user@hive.apache.org Subject: Re: How to check if all key/value pairs in two maps are equal? Can you cast it into string and compare? On Thu, Feb 2, 2012 at 9:18 AM, Sunderlin, Mark mailto:mark.sunder...@teamaol.com>> wrote: I am trying to see of two hive maps have the same data in

How to check if all key/value pairs in two maps are equal?

2012-02-02 Thread Sunderlin, Mark
I am trying to see of two hive maps have the same data in them. I am not looking to see if any single key-value pair in the maps match, I am looking to see if a)There is a one to one match between the keys in map1 and the keys in map2 b) The matching keys have the same value in both map

RE: Hive ODBC - Microsoft’s Involvement

2012-02-02 Thread Sunderlin, Mark
>> Any reason you want to use a ODBC and not Thrift ? Hive supports the thrift >> protocol. There are thrift libraries for C# and you can easily integrate it >> into your project for direct access to HIVE via your C# code. ODBC opens Hive up for use through any number of Windows based SQL clien

RE: Hive Security

2012-02-01 Thread Sunderlin, Mark
We only allow the anonymous connection via JDBC from a specific host designated to run application jobs against Hive. For end user connections, we use Cloudera’s Hue interface which has specific user ID/password level authentication. It’s not a bad web based client tool at all. Not as robust

RE: Question on regexp_extract() , Index?

2011-10-27 Thread Sunderlin, Mark
Ah, easy enough! And if you have no groups, you just want the match, it seems index of 0 works just fine. select regexp_extract('junk:text:ua123','ua[0-9]+',0) from dual -- assumes you have created a dummy hive table called dual ua123 --- Mark E. Sunderlin Solutions Architect |AOL Data Wareho

Question on regexp_extract() , Index?

2011-10-27 Thread Sunderlin, Mark
I've been working with the hive regexp_extract(string subject, string pattern, int index) command. In the hive language manual, https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-StringFunctions, the following description for this function is given: Returns t

a query to return the keys in a hive map?

2011-06-28 Thread Sunderlin, Mark
https://issues.apache.org/jira/browse/HIVE-1734. Did this or an equivalent functionality to return the keys of a key/value pair get implemented into Hive 0/70? If not, does anyone have a hack or clever query that is a work around? "I have a clever and cunning query mi-lord" ;-) --- Mark E. Su

Firefox 4.1 and Hue: Problems?

2011-06-15 Thread Sunderlin, Mark
Is anyone else having problems with Hue based on Hive 07.0 and Firefox 4.1? My Firefox browser crashes every time I run, but the dreaded I 8.0 runs fine. Any idea where I start? --- Mark E. Sunderlin Solutions Architect |AOL Data Warehouse P: 703-256-6935 | C: 540-327-6222 AIM: MESunderlin 220

Selecting an entire map, not just one element with Squirrel?

2011-04-28 Thread Sunderlin, Mark
OK, feeling a bit dumb here . so I need the hive user group jolt to the head ... Given a table like: describe hive_map_test col_namedata_type comment log_record_type int key_pairs map ev_date string and given that this works: sel

End User Clients for Hive?

2011-03-18 Thread Sunderlin, Mark
As we prepare Hive for use by general business analysts and other end-users, I am wondering what the community's experience is with clients for end users? My user base currently is using tools such as WinSQL and Toad on their Windows machines to access current systems, such as MySQL, Oracle and

RE: Hadoop error 2 while joining two large tables

2011-03-16 Thread Sunderlin, Mark
hadoop n00b asks, "Is adding more nodes the solution to such problem?" Whatever else answers you get, you should append " ... and add more nodes." More nodes is never a bad thing ;-) --- Mark E. Sunderlin Solutions Architect |AOL Data Warehouse P: 703-256-6935 | C: 540-327-6222 AIM: MESunderlin

Associative Arrays in Hive?

2011-03-02 Thread Sunderlin, Mark
Let us say my log data that I want to place a log file into hive. And the log file itself looks something like this: Event_time, event_type, event_data_blob And the blob data looks like "Key1=value1;key2=value2;key3=value3 ... keyn=valuen" This looks like maybe I start like this: Create table

RE: Hive Not Returning Column Names, even what not using 'When'??

2011-02-18 Thread Sunderlin, Mark
l Regards, Peter Sirota From: Sunderlin, Mark [mailto:mark.sunder...@teamaol.com] Sent: Thursday, February 17, 2011 1:00 PM To: user@hive.apache.org Subject: Hive Not Returning Column Names, even what not using 'When'?? When using either Hue or using Squirrel and jdbc, I get "_co

Hive Not Returning Column Names, even what not using 'When'??

2011-02-17 Thread Sunderlin, Mark
When using either Hue or using Squirrel and jdbc, I get "_col0, _col1, _col2 ..." as my columns names when querying my hive tables. I see in https://issues.apache.org/jira/browse/HIVE-1346 that this is a known issue when using the where clause, but I get in circumstances other than using a wher