Read about hive partitions and bucketing. Since your location has multiple directories, Hive needs to know how to traverse them..
Hope this helps. On May 19, 2016 5:51 AM, "Abhishek Dubey" <abhishek.du...@xoriant.com> wrote: > Hi, > > > > In hdfs I have a directory structure like this. > > /user/hdfs/Data/Data1/File1 > > /user/hdfs/Data/Data2/File2 > > > > And I am creating an external table like: > > CREATE external TABLE db.tablename > > ( > > amt1 STRING, > > amt2 STRING, > > amt3 STRING > > ) > > row format delimited > > fields terminated by ',' > > location '/user/hdfs/Data/'; > > > > Also, I have set two properties: > > set mapred.input.dir.recursive=true; > > set hive.mapred.supports.subdirectories=true; > > > > This setup is working perfectly fine on my local single node vm, Having > all vanilla apache installations and setup, > > > > But. on cloudera 5.3.3 cluster of 4 nodes, above mentioned properties for > recursive lookup of sub directories for an external hive table is not > working. > > In the Cloudera manager i have added the properties in Hive-Site.xml, > deployed configuration and restarted Hive service but still not working. > > <property> > > <name> mapred.input.dir.recursive</name> > > <value>true</value> > > </property> > > <property> > > <name>hive.mapred.supports.subdirectories</name> > > <value>true</value> > > </property> > > > > When querying select * on CDH What i get is this, Zero rows. > > hive> select * from tablename; > > OK > > Time taken: 0.322 seconds > > hive> > > > > Whereas on local vm it is giving desired output. > > > > Is there anything else on CDH that we need to take care to pick data from > subdirectories into hive table? > > > > Thanks in advance. > *Abhishek Dubey* > > > > >
