It is a hadoop limitation. hdfs move operation is inexpensive. I am assuming that is not an option to you because you want to save the path structure (for some backward compatibility sake).
Something like symbolic links (i think its not supported in 0.20, not sure) or path filter might help. But, it would be a hack. Thanks, Aniket On Thu, Dec 8, 2011 at 3:59 PM, Jasper Knulst <jasper.knu...@incentro.com>wrote: > Hi Vince, > > Hive partitioning can only exist by issueing new directories in HDFS. > There is no way to partition the data in a Hive table without adding extra > filepaths/dirs in HDFS. > > For an external table you have to redistribute the data yourself in > corresponding filepaths and add the new partition based on that by editing > the Hive metadata. > > Cheers Jasper > Op 8 dec. 2011 21:47 schreef "Vince Hoang" <vho...@cafepress.com> het > volgende: > > Hi, >> >> I am running Hive 0.7.0 with Hadoop 0.20.2. I have one HDFS folder >> full of web server logs dated back several months. >> >> Is possible to partition an EXTERNAL TABLE without copying/moving files >> or altering the layout of the directory? >> >> For example, in HDFS, I have: >> >> > /logs/log-2011-09-01 >> > /logs/log-2011-09-02 >> > … >> > /logs/log-2011-12-01 >> >> I'd like to know if it's possible to partition the EXTERNAL TABLE by >> date without having to create subdirectories: >> >> > /logs/2011-09-01/log-2011-09-01 >> > /logs/2011-09-02/log-2011-09-02 >> > … >> > /logs/2011-12-01/log-2011-12-01 >> >> Is it possible? >> >> Thanks, >> Vince >> >> >> The contents of this message, together with any attachments, are intended >> only for the use of the individual or entity to which they are addressed >> and may contain information that is confidential and exempt from >> disclosure. If you are not the intended recipient, you are hereby notified >> that any dissemination, distribution, or copying of this message, or any >> attachment, is strictly prohibited. If you have received this message in >> error, please notify the original sender immediately by telephone or by >> return E-mail and delete this message, along with any attachments, from >> your computer. Thank you. >> > -- "...:::Aniket:::... Quetzalco@tl"
