does this happen for 1 specific partition or all of them?
On 10/25/16, 12:47 AM, "aft" <aft...@gmail.com> wrote:
>Hi,
>
>Table created with this :
>
> $hive>create table syslog_staged (id string, facility string,
>sender string, severity string, tstamp string, service string, msg
>string) partitioned by (hostname string, year string, month string,
>day string) clustered by (id) into 20 buckets stored as orc
>tblproperties("transactional"="true");
>
>the table is populated with Apache nifi's PutHiveStreaming...
>
> $hive>alter table syslog_staged partition
>(hostname="cloudserver19", year="2016", month="10", day="24") compact
>'major';
>
>Now it turns out compaction fails for some reason.....(from job history)
>
> No of maps and reduces are 0 job_1476884195505_0031
> Job commit failed: java.io.FileNotFoundException: File
>hdfs://hadoop1.openstacksetup.com:8020/apps/hive/warehouse/log.db/syslog_s
>taged/hostname=cloudserver19/year=2016/month=10/day=24/_tmp_27c40005-658e-
>48c1-90f7-2acaa124e2fa
>does not exist.
> at
>org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(Distribute
>dFileSystem.java:904)
> at
>org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSys
>tem.java:113)
> at
>org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSyst
>em.java:966)
> at
>org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSyst
>em.java:962)
> at
>org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver
>.java:81)
> at
>org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSys
>tem.java:962)
> at
>org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorOutputCommitt
>er.commitJob(CompactorMR.java:776)
> at
>org.apache.hadoop.mapred.OutputCommitter.commitJob(OutputCommitter.java:29
>1)
> at
>org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProce
>ssor.handleJobCommit(CommitterEventHandler.java:285)
> at
>org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProce
>ssor.run(CommitterEventHandler.java:237)
> at
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1142)
> at
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:617)
>
>from hive metastore log :
>
> 2016-10-24 16:33:35,503 WARN [Thread-14]: compactor.Initiator
>(Initiator.java:run(132)) - Will not initiate compaction for
>log.syslog_staged.hostname=cloudserver19/year=2016/month=10/day=24
>since last hive.compactor.initiator.failed.compacts.threshold attempts
>to compact it failed.
>
>
>Hive version:
>1.2.1000
>
