I have a problem reading parquet files. sql: select count(1) from omega.dwd_native where year='2016' and month='07' and day='05' and hour='12' and appid='6'; The hive partition is (year,month,day,appid)
only two tasks, and it will list all directories in my table, not only /user/omega/events/v4/h/2016/07/07/12/appid=6 [Stage 1:> (0 + 0) / 2] 16/07/08 16:16:51 INFO sources.HadoopFsRelation: Listing hdfs://mycluster-tj/user/omega/events/v4/h/2016/05/31/21/appid=1 16/07/08 16:16:51 INFO sources.HadoopFsRelation: Listing hdfs://mycluster-tj/user/omega/events/v4/h/2016/06/28/20/appid=2 16/07/08 16:16:51 INFO sources.HadoopFsRelation: Listing hdfs://mycluster-tj/user/omega/events/v4/h/2016/07/22/21/appid=65537 16/07/08 16:16:51 INFO sources.HadoopFsRelation: Listing hdfs://mycluster-tj/user/omega/events/v4/h/2016/08/14/05/appid=65536
