RE: Hive indexing optimization

2015-06-30 Thread Bennie Leo
Thank you, I will do that. B Subject: Re: Hive indexing optimization From: jpullokka...@hortonworks.com To: user@hive.apache.org Date: Tue, 30 Jun 2015 18:46:50 + Index doesn’t seems to be kicking in this case. Please file a bug for this. Thanks John From: Bennie Leo Reply-To

Re: Hive indexing optimization

2015-06-30 Thread John Pullokkaran
PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: RE: Hive indexing optimization I've attached the output. Thanks. B Subject: Re: Hive indexing optimization From: jpullokka...@horton

RE: Hive indexing optimization

2015-06-29 Thread Bennie Leo
I've attached the output. Thanks. B Subject: Re: Hive indexing optimization From: jpullokka...@hortonworks.com To: user@hive.apache.org Date: Mon, 29 Jun 2015 19:17:44 + Could you post explain extended output? From: Bennie Leo Reply-To:

Re: Hive indexing optimization

2015-06-29 Thread John Pullokkaran
:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: RE: Hive indexing optimization Here is the explain output: STAGE PLANS: Stage: Stage-1 Tez Edges: Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE) Vertices: Map 1

RE: Hive indexing optimization

2015-06-29 Thread Bennie Leo
putFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Stage: Stage-0 Fetch Operator limit: -1 Thank you, B > Subject: Re: Hive indexing optimiz

Re: Hive indexing optimization

2015-06-27 Thread John Pullokkaran
"SELECT StartIp, EndIp, Country FROM ipv4geotable” should have been rewritten as a scan against index table. BitMap Indexes seems to support inequalities (<=, <, >=). Post the explain plan. On 6/26/15, 8:56 PM, "Gopal Vijayaraghavan" wrote: >Hi, > >Hive indexes won¹t really help you speed up t

Re: Hive indexing optimization

2015-06-26 Thread Gopal Vijayaraghavan
Hi, Hive indexes won¹t really help you speed up that query right now, because of the plan it generates due to the <= clauses. > CREATETABLE ipv4table > AS > SELECT logon.IP, ipv4.Country > FROM > (SELECT * FROM logontable WHERE isIpv4(IP)) logon > LEFT OUTER JOIN > (SELECT StartIp, EndIp, Country

RE: Hive indexing optimization

2015-06-26 Thread Bennie Leo
veCompactIndexInputFormat;" ? I don't know how I could include this within my current query. Cheers, B Subject: Re: Hive indexing optimization From: jpullokka...@hortonworks.com To: user@hive.apache.org Date: Fri, 26 Jun 2015 01:27:21 + Set hive.optimize.index.filter=tru

Re: Hive indexing optimization

2015-06-25 Thread John Pullokkaran
e.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Hive indexing optimization Hi, I am attempting to optimize a query using indexing. My current query converts an ipv4 address to a country using a geolocation table. However, the geolocation table is fairly large and

Hive indexing optimization

2015-06-25 Thread Bennie Leo
Hi, I am attempting to optimize a query using indexing. My current query converts an ipv4 address to a country using a geolocation table. However, the geolocation table is fairly large and the query takes an impractical amount of time. I have created indexes and set the binary search param

Hive Indexing and ORC

2014-09-06 Thread Alain Petrus
Hello, Is it possible to create an index on table stored as ORC and compressed as Snappy? Does it make sense? I am wondering if Hive indexing is a mature functionality? Thanks, Alain

Hive Indexing and ORC

2014-09-06 Thread Alain Petrus
Hello, Is it possible to create an index on table stored as ORC and compressed as Snappy? Does it make sense? I am wondering if Hive indexing is a mature functionality? Thanks, Alain

Hive Indexing and ORC

2014-09-06 Thread Alain Petrus
Hello, Is it possible to create an index on table stored as ORC and compressed as Snappy? Does it make sense? I am wondering if Hive indexing is a mature functionality? Thanks, Alain

Help me understand Hive indexing.

2013-11-06 Thread Heller, Chris
Hi, I am new to Hive, and am trying to setup an index on a Hive table to improve query performance. I am presently using the CDH 4.2 Hadoop distribution, which ships with Hive 0.10, so from what I have read table index support should be available. What I am seeing though is that when I go and cre

Review & improvement request: Hive indexing doc

2013-06-28 Thread Lefty Leverenz
The stub of an Indexing user doc in the Hive wiki's Language Manual now includes some simple examples, adapted from the test suite. Would someone who uses Hive indexes please review it and make any necessary corrections & additions? For example, I omitted examples of indexes on partitioned tables

hive indexing

2011-09-30 Thread Shouguo Li
hi, i'm looking at adding indexes to our hive tables, am wondering if anyone can share some thoughts on it. is there a performance/space trade off comparison or metrics? obviously it would be costly to index all columns in your tables, so what types of columns are worth indexing? thx!