On Wed, May 15, 2013 at 3:38 AM, Peter Marron < peter.mar...@trilliumsoftware.com> wrote:
> Hi,**** > > ** ** > > I’m using Hive 0.10.0 and Hadoop 1.0.4.**** > > ** ** > > I would like to create a normal table but have some of my code run so that > I can remove filtering**** > > parts of the query and limit the output in the splits of the InputFormat. > I believe that this is**** > > “Filtering Pushdown” as described in > https://cwiki.apache.org/Hive/filterpushdowndev.html**** > > I have tried various approaches and run into problems and I was wondering* > *** > > if anyone had any suggestions as to how I might proceed. > I've started doing similar work for the ORC reader. > **** > > ** ** > > Firstly although that page mentions InputFormat there doesn’t seem to be > any way (that I can find)**** > > to perform filter passing to InputFormats and so I gave up on that > approach. > There is. You just need to set hive.optimize.index.filter to true. See https://issues.apache.org/jira/browse/HIVE-4242. That said, we really need to create a better interface that allows inputformats to negotiate what parts of the predicate they can process. -- Owen > ** >
