Re: Query crawls through reducer

2013-03-24 Thread Keith Wiley
Thanks. On Mar 22, 2013, at 21:02 , Nitin Pawar wrote: > instead of >= can you just try = if you want to limit top 100 (b being a > partition i guess it will have more that 100 records to fit into your limit) > > to improve your query performance your table file format matters as well. > Whi

Re: Query crawls through reducer

2013-03-22 Thread Nitin Pawar
instead of >= can you just try = if you want to limit top 100 (b being a partition i guess it will have more that 100 records to fit into your limit) to improve your query performance your table file format matters as well. Which one are you using? how many partitions are there? what's the size

Query crawls through reducer

2013-03-22 Thread Keith Wiley
The following query translates into a many-map-single-reduce job (which is common) and also slags through the reduce stage...it's killing the overall query: select * from a where b >= 'c' order by b desc limit 100 Note that b is a partition. What component is making the reducer heavy? Is it