Re: Verifying that a query uses orc bloom filters, orc storage indexes

2015-07-30 Thread Yue Liu
First, you need to know the total record number of your table. After processing the query, then you can get the input record number of MapReduce job from the counter "RECORDS_IN" of Hive or "Map input records" of Map-Reduce Framework. If the input number is much smaller than the total number, it s

Re: Verifying that a query uses orc bloom filters, orc storage indexes

2015-07-30 Thread Prasanth Jayachandran
If you are using tez, you can verify that using counters that gets printed after query execution. You need set hive.tez.exec.print.summary=true for tez to print counters after execution. Thanks Prasanth On Jul 30, 2015, at 9:31 AM, Jörn Franke mailto:jornfra...@gmail.com>> wrote: Hi, Is the

Verifying that a query uses orc bloom filters, orc storage indexes

2015-07-30 Thread Jörn Franke
Hi, Is there any official way to verify that a query leveraged orc bloom filters or orc indexes? For example, number of bytes (rows) not processed thanks to bloom filters or storage indexes? Some indicators in the explain output? Thank you. Best reagrds