Re: Indexing Help

2011-08-05 Thread Shouguo Li
on a side note, i'm looking at adding indexes to our hive tables as well, is there a performance/space trade off comparison or metrics? thx! On Wed, Aug 3, 2011 at 10:52 AM, Siddharth Ramanan < siddharth.rama...@gmail.com> wrote: > Hi all, > I have used compact index for my table and th

Indexing Help

2011-08-03 Thread Siddharth Ramanan
Hi all, I have used compact index for my table and the response time is same for a query with as well as without index now. Previously, it was showing improvement. I just changed some parameters to increase heap size and then it is behaving weird. so, how can I make sure that my query is u

Re: Indexing help

2011-08-01 Thread Siddharth Ramanan
The reduce percentage keeps fluctuating when, the alter index command is being keyed. The logs just give " out of memory error " after tweaking some properties, the earlier exceptions doesn't appear now. Can anyone guide me here? I have increased the heap space upto 4gb.. still, getting the same ex

Re: Indexing help

2011-07-28 Thread Siddharth Ramanan
Hi, I am adding the log information for a reduce task. I am running hadoop in standalone mode. 2011-07-28 19:16:42,621 ERROR org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during JDBC connection to jdbc:derby:;databaseName=TempStatsStore;create=true. java.lang.ClassNotFoundExc

Indexing help

2011-07-28 Thread Siddharth Ramanan
Hi, I have a table, which has close to a billion rows.. I am trying to create an index for the table, when I do the alter command, I always end up with map-reduce jobs with errors. The same runs fine for small tables though, I also notice that the number of reducers are set to 24, even if set