Re: ELB Log processing

2016-09-20 Thread Manish Rangari
'QUERY','tid') as tid, parse_url(url, 'QUERY','eid') as eid, > parse_url(url, 'QUERY','did') as did, protocol, useragent, ssl_cipher, > ssl_protocol > > from elblog; > > > > Dudu > > > > *From:* Markovitz, Dudu [

RE: ELB Log processing

2016-09-20 Thread Markovitz, Dudu
mailto:dmarkov...@paypal.com] Sent: Tuesday, September 20, 2016 6:06 PM To: user@hive.apache.org Subject: RE: ELB Log processing create view elb_raw_log_detailed as select request_date, elbname, requestip, requestport, backendip, backendport, requestprocessingtime, backendprocessingtime, cli

RE: ELB Log processing

2016-09-20 Thread Markovitz, Dudu
0, 2016 4:09 PM To: user@hive.apache.org Subject: Re: ELB Log processing Guys, I am struggling to create this view. I am keep getting the error in bold. I found that I need to use lateral view but still I am not able to get the syntax right. hive> create view elb_raw_log_detailed as select req

Re: ELB Log processing

2016-09-20 Thread Manish Rangari
Guys, I am struggling to create this view. I am keep getting the error in bold. I found that I need to use lateral view but still I am not able to get the syntax right. hive> create view elb_raw_log_detailed as select request_date, elbname, requestip, requestport, backendip, backendport, requestp

Re: ELB Log processing

2016-09-20 Thread Manish Rangari
Yes views looks like a way to go On Tue, Sep 20, 2016 at 3:49 PM, Damien Carol wrote: > The royal way to do that is a view IMHO. > > 2016-09-20 12:14 GMT+02:00 Manish Rangari > : > >> Thanks for the reply Damien. The suggestion you gave is really useful. >> Currently I am achieving my desired o

Re: ELB Log processing

2016-09-20 Thread Damien Carol
The royal way to do that is a view IMHO. 2016-09-20 12:14 GMT+02:00 Manish Rangari : > Thanks for the reply Damien. The suggestion you gave is really useful. > Currently I am achieving my desired output by performing below steps. But I > want to achieve the desired result in one step instead of t

Re: ELB Log processing

2016-09-20 Thread Manish Rangari
Thanks for the reply Damien. The suggestion you gave is really useful. Currently I am achieving my desired output by performing below steps. But I want to achieve the desired result in one step instead of two. Do we have any way so that I can get the aid, did etc in create table statement? If not I

Re: ELB Log processing

2016-09-20 Thread Damien Carol
see the udf *parse_url_tuple* SELECT b.* FROM src LATERAL VIEW parse_url_tuple(fullurl, 'HOST', 'PATH', 'QUERY', 'QUERY:id') b as host, path, query, query_id LIMIT 1; https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-parse_url_tuple 2016-09-20 11:22 GMT+02:00

ELB Log processing

2016-09-20 Thread Manish Rangari
Guys, I want to get the field of elb logs. A sample elb log is given below and I am using below create table definition. It is working fine. I am getting what I wanted but now I want the bold part as well. For example eid, tid, aid. Can anyone help me how can I match them as well. NOTE: The posit