[
https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242932#comment-13242932
]
Phabricator commented on HIVE-2711:
-----------------------------------
ashutoshc has commented on the revision "HIVE-2711 [jira] Make the header of
RCFile unique".
It seems like you are backward compatible with SEQ6 but not for anything
before that. If the intent is to break backward compatibility, then I think we
should send an email on both dev and user list about this change, since folks
might have historical data in this format, which can't be read then.
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1205 By doing this, you
are effectively dropping the ability to read data before version SEQ6. This is
backward incompatible change.
ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1224 By removing this,
you will not be able to read data before version SEQ2.
ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1252 This removes
ability to read before version 6.
ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1256 This removes
ability to read before SEQ1
ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1357 This removes
ability to read before SEQ1
REVISION DETAIL
https://reviews.facebook.net/D2115
> Make the header of RCFile unique
> --------------------------------
>
> Key: HIVE-2711
> URL: https://issues.apache.org/jira/browse/HIVE-2711
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: HIVE-2711.D2115.1.patch
>
>
> The RCFile implementation was copied from Hadoop's SequenceFile and copied
> the 'magic' string in the header. This means that you can't use the header to
> distinguish between RCFiles and SequenceFiles.
> I'd propose that we create a new header for RCFiles (RCF?) to replace the
> current SEQ. To maintain compatibility, we'll need to continue to accept the
> current 'SEQ\06' and just make new files contain the new header.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
