Re: RegexSerDe with Filters

2016-07-14 Thread Arun Patel
ngHandler Message%' then val_num end))as timestamps_no_dup > > > > ... > > fromv > > group bytid > > ; > > > > *From:* Arun Patel [mailto:arunp.bigd...@gmail.com] > *Sent:* Sunday, July 03, 2016 12:39 AM > > *To:* user@h

Re: RegexSerDe with Filters

2016-07-02 Thread Arun Patel
;,min (case when att like '%ProcessingHandler Message%' then > val_numend) as timestamp > >,min (case when att = 'Total time' then > val_numend) as Total_time > > > > fromv > > &g

RE: RegexSerDe with Filters

2016-07-02 Thread Markovitz, Dudu
then val_numend) as Total_time fromv group bytid ; From: Arun Patel [mailto:arunp.bigd...@gmail.com] Sent: Friday, July 01, 2016 9:20 PM To: user@hive.apache.org Subject: Re: RegexSerDe with Filters Dudu, Thanks for your continued support. I need one mor

Re: RegexSerDe with Filters

2016-07-01 Thread Arun Patel
]{3})\s+(\S+)\s+\[([^]]+)\]\s+(\S+)\s+:\s+(TID:\s\d+)?\s*(.*) >> >> >> >> I’ll send you a screen shot in private, since you don’t want to expose >> the data. >> >> >> >> Dudu >> >> >> >> >> >> *From:* Arun Pa

Re: RegexSerDe with Filters

2016-06-24 Thread Arun Patel
^]]+)\]\s+(\S+)\s+:\s+(TID:\s\d+)?\s*(.*) > > > > I’ll send you a screen shot in private, since you don’t want to expose the > data. > > > > Dudu > > > > > > *From:* Arun Patel [mailto:arunp.bigd...@gmail.com] > *Sent:* Friday, June 24, 2016 9:33 PM > &

RE: RegexSerDe with Filters

2016-06-24 Thread Markovitz, Dudu
From: Arun Patel [mailto:arunp.bigd...@gmail.com<mailto:arunp.bigd...@gmail.com>] Sent: Wednesday, June 22, 2016 2:57 AM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: Re: RegexSerDe with Filters Thank you very much, Dudu. This really helps. On Tue, Jun 21, 2016 at

Re: RegexSerDe with Filters

2016-06-24 Thread Arun Patel
> > > *From:* Arun Patel [mailto:arunp.bigd...@gmail.com] > *Sent:* Wednesday, June 22, 2016 2:57 AM > *To:* user@hive.apache.org > *Subject:* Re: RegexSerDe with Filters > > > > Thank you very much, Dudu. This really helps. > > > > On Tue, Jun 21, 2016 at

RE: RegexSerDe with Filters

2016-06-23 Thread Markovitz, Dudu
My pleasure. Please feel free to reach me if needed. Dudu From: Arun Patel [mailto:arunp.bigd...@gmail.com] Sent: Wednesday, June 22, 2016 2:57 AM To: user@hive.apache.org Subject: Re: RegexSerDe with Filters Thank you very much, Dudu. This really helps. On Tue, Jun 21, 2016 at 7:48 PM

Re: RegexSerDe with Filters

2016-06-21 Thread Arun Patel
et_request; > > > > create view log_v_get_request > > as > > select c1 > >,ts > >,log_rec_level > >,c4 > >,c5 > >,tid > >,substr (txt,31) as path > > >

RE: RegexSerDe with Filters

2016-06-21 Thread Markovitz, Dudu
,tid ,regexp_extract (txt,'rowkey (\\S+)',1) as rowkey ,regexp_extract (txt,'lock id (\\S+)',1) as lock_id fromlog_V where txt like 'Unlock request for schema DU %' ; From: Markovitz, Dudu [mailto:dmarkov...@paypal.com]

RE: RegexSerDe with Filters

2016-06-21 Thread Markovitz, Dudu
: user@hive.apache.org Subject: RegexSerDe with Filters Hello Hive Experts, I use flume to ingest application specific logs from Syslog to HDFS. Currently, I grep the HDFS directory for specific patterns (for multiple types of requests) and then create reports. However, generating reports for

RegexSerDe with Filters

2016-06-20 Thread Arun Patel
Hello Hive Experts, I use flume to ingest application specific logs from Syslog to HDFS. Currently, I grep the HDFS directory for specific patterns (for multiple types of requests) and then create reports. However, generating reports for Weekly and Monthly are not salable. I would like to create