I think you have an extra '/' in the HDFS URI Daniel
> On 30 באפר׳ 2015, at 16:46, Harsha N <[email protected]> wrote: > > Thanks for your reply, > > analyze table table1 partition (dt=201501) compute statistics;--returns the > same error > Total jobs = 1 > Launching Job 1 out of 1 > Number of reduce tasks is set to 0 since there's no reduce operator > java.io.IOException: cannot find dir = hdfs:///data/dt =201501/1430201400/ > in pathToPartitionInfo: [hdfs:/data/dt=201501/1430201400/] > at > org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:347) > > To Add On > I am working on External tables in Hive 0.13.1-cdh5.3.2 > > -Harsha > >> On Thu, Apr 30, 2015 at 12:37 AM, Mich Talebzadeh <[email protected]> >> wrote: >> Hi Harsha, >> >> >> >> Have you updated stats on table1 after partition adding? In other words it >> is possible that the optimiser is not aware of that partition yet? >> >> >> >> analyze table table1 partition (dt=201501) compute statistics; >> >> >> >> HTH >> >> >> >> Mich Talebzadeh >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> >> Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", >> ISBN 978-0-9563693-0-7. >> >> co-author "Sybase Transact SQL Guidelines Best Practices", ISBN >> 978-0-9759693-0-4 >> >> Publications due shortly: >> >> Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and >> Coherence Cache >> >> Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume >> one out shortly >> >> >> >> NOTE: The information in this email is proprietary and confidential. This >> message is for the designated recipient only, if you are not the intended >> recipient, you should destroy it immediately. Any information in this >> message shall not be understood as given or endorsed by Peridale Ltd, its >> subsidiaries or their employees, unless expressly so stated. It is the >> responsibility of the recipient to ensure that this email is virus free, >> therefore neither Peridale Ltd, its subsidiaries nor their employees accept >> any responsibility. >> >> >> >> From: Harsha N [mailto:[email protected]] >> Sent: 30 April 2015 07:24 >> To: [email protected] >> Subject: Hive Alter Partition Location Issue >> >> >> >> Hi All, >> >> Can experts share your view on Hive behaviour in below scenario. >> >> >> >> I am facing below issue on using alter partition locations in hive. >> >> >> >> select count(*) from table1 where dt = 201501; >> >> >> >> Total jobs = 1 >> >> Launching Job 1 out of 1 >> >> Number of reduce tasks determined at compile time: 1 >> >> In order to change the average load for a reducer (in bytes): >> >> set hive.exec.reducers.bytes.per.reducer=<number> >> >> In order to limit the maximum number of reducers: >> >> set hive.exec.reducers.max=<number> >> >> In order to set a constant number of reducers: >> >> set mapreduce.job.reduces=<number> >> >> java.io.IOException: cannot find dir = hdfs:///data/dt =201501/1430201400/ >> >> in pathToPartitionInfo: [hdfs:/data/dt=201501/1430201400/] >> >> >> >> Below are the steps I have followed. >> >> I have altered a partition location in hive using below command. >> >> ALTER TABLE table1 PARTITION (dt=201501) SET LOCATION >> 'hdfs:///data/dt=201501/1430201400/'; >> >> >> >> I have inserted new data into this new location. >> >> >> >> INSERT INTO TABLE table1 >> >> SELECT * FROM table2 where dt=201501 >> >> >> >> select count(*) from table1 where dt = 201501; doesn't work but >> >> select * from table1 where dt = 201501 works good. >> >> >> >> Please let me know if you need more information. >> >> >> >> Thanks >> >> Harsha >> >
