Ryan Pifer created HUDI-2443:
--------------------------------
Summary: KVComparator in HFile for metadata table is tied to HBase
version
Key: HUDI-2443
URL: https://issues.apache.org/jira/browse/HUDI-2443
Project: Apache Hudi
Issue Type: Improvement
Reporter: Ryan Pifer
There is no guarantee of compatibility between HFile reading/writing from
different versions of ** HBase. For instance, HBase version 2.2.6 modified the
comparator class stored in HFile metadata to
{{org.apache.hadoop.hbase.KeyValue.KVComparator}} and
{{org.apache.hadoop.hbase.KeyValue$KeyComparator}} no longer exists which may
create issues when reading HFiles created by earlier versions of Hudi. And this
comparator class is already marked as deprecated for {{CellComparatorImpl.}}
{{This also creates issues when shading HBase dependencies under Hudi
namespace}}
{{Can we refactor HoodieKVComparator which is used in bootstrap index and have
it applied for all use of HFile in Hudi. }}
{{[https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/bootstrap/index/HFileBootstrapIndex.java#L580-L586]}}
There will need to be some logic as well for backward compatibility when
reading metadata which was written from earlier version of Hudi as well. We can
maybe enforce that user must perform an upgrade step beforehand which re-writes
metadata base file
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
