Hi,
We use HCatWriter to write records to Hive, and I've recently run into a
problem with HCatWriter that seems intractable.
We can write tables without partitions all the live-long day, but any attempt
to write to a partition results in the following error:
"net/redpoint/hiveclient/DMHCatWriter.closeWriter:org.apache.hive.hcatalog.common.HCatException
: 2004 : HCatOutputFormat not initialized, setOutput has to be called. Cause :
org.apache.hive.hcatalog.common.HCatException : 2006 : Error adding partition
to metastore. Cause : org.apache.hadoop.security.AccessControlException:
Permission denied. user=nbamford is not the owner of inode=state=CO"
Digging into the source for
org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer.constructPartition,
I find the following lines:
for (FieldSchema partKey : table.getPartitionKeys()) {
if (i++ != 0) {
fs.mkdirs(partPath); // Attempt to make the path in case it does not exist
before we check
applyGroupAndPerms(fs, partPath, perms, grpName, false);
}
partPath = constructPartialPartPath(partPath, partKey.getName().toLowerCase(),
partKVs);
}
}
The error is thrown from the applyGroupAndPerms function, which you will note
does not check for the directory existing, and having the right permissions (in
this case, it does).
I am at a complete loss for how to proceed. I can't even think of a
workaround. It seems to me HCatWriter simply cannot write partitions when
Sentry and the HDFS ACL plugin are in force.
