[
https://issues.apache.org/jira/browse/HIVE-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-8966:
-----------------------------
Attachment: HIVE-8966.2.patch
A new version of the patch that moves Jihong's code into AcidUtils.getAcidState
so that delta directories with flush length files are not put into the list of
files to compact.
[~jihongliu], could you test this on your end to make sure it addresses your
issues. I'll also do some long running tests to see that it allows compaction
while streaming is ongoing.
> Delta files created by hive hcatalog streaming cannot be compacted
> ------------------------------------------------------------------
>
> Key: HIVE-8966
> URL: https://issues.apache.org/jira/browse/HIVE-8966
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
> Affects Versions: 0.14.0
> Environment: hive
> Reporter: Jihong Liu
> Assignee: Alan Gates
> Priority: Critical
> Fix For: 0.14.1
>
> Attachments: HIVE-8966.2.patch, HIVE-8966.patch
>
>
> hive hcatalog streaming will also create a file like bucket_n_flush_length in
> each delta directory. Where "n" is the bucket number. But the
> compactor.CompactorMR think this file also needs to compact. However this
> file of course cannot be compacted, so compactor.CompactorMR will not
> continue to do the compaction.
> Did a test, after removed the bucket_n_flush_length file, then the "alter
> table partition compact" finished successfully. If don't delete that file,
> nothing will be compacted.
> This is probably a very severity bug. Both 0.13 and 0.14 have this issue
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
