[
https://issues.apache.org/jira/browse/HIVE-25521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426356#comment-17426356
]
Panagiotis Garefalakis commented on HIVE-25521:
-----------------------------------------------
Resolved via https://github.com/apache/hive/pull/2639
> Data corruption when concatenating files with different compressions in same
> table/partition
> --------------------------------------------------------------------------------------------
>
> Key: HIVE-25521
> URL: https://issues.apache.org/jira/browse/HIVE-25521
> Project: Hive
> Issue Type: Bug
> Reporter: Harish JP
> Assignee: Harish JP
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Currently if files of different compressions are in same directory then
> concatenate can fail and cause data corruption. This happens because file can
> be moved by one task as incompatible file and the other tasks will fail after
> this.
>
> This issue is addressed in this Jira by only processing a file in one task
> where offset 0 is process and ignoring the the file in all other tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
