Hi Koert
As you are creating dir/sub dirs using mapreduce jobs out of hive, hive
is unaware of these sub dirs. There is no other way in such cases other than an
add partition DDL to register the dir with a hive partition.
If you are using oozie or shell to trigger your jobs,you can accomplish it as
-use java to come up with the correct add partition statement and write those
statement(s) into a file
-execute the file using hive -f <fileName>
Hope it helps!..
Regards
Bejoy K S
From handheld, Please excuse typos.
-----Original Message-----
From: Koert Kuipers <ko...@tresata.com>
Date: Wed, 8 Feb 2012 11:04:18
To: <user@hive.apache.org>
Reply-To: user@hive.apache.org
Subject: external partitioned table
hello all,
we have an external partitioned table in hive.
we add to this table by having map-reduce jobs (so not from hive) create
new subdirectories with the right format (partitionid=partitionvalue).
however hive doesn't pick them up automatically. we have to go into hive
shell and run "alter table sometable add partition
(partitionid=partitionvalue)". to make matter worse hive doesnt really lend
itself to running such an add-partition-operation from java (or for that
matter: hive doesn't lend itself to any easy programmatic manipulations...
grrr. but i will stop now before i go on a a rant).
any suggestions how to approach this? thanks!
best, koert
