Vova Kolmakov created HUDI-9408:
-----------------------------------
Summary: Enable TestSparkHoodieHBaseIndex for Spark 4
Key: HUDI-9408
URL: https://issues.apache.org/jira/browse/HUDI-9408
Project: Apache Hudi
Issue Type: Improvement
Reporter: Vova Kolmakov
TestSparkHoodieHBaseIndex fails on Spark 4 with error:
{code:java}
java.lang.NoSuchFieldError: NVDIMM
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockStoragePolicySuite.createDefaultSuite(BlockStoragePolicySuite.java:68)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.<init>(BlockManager.java:531)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:926)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:851)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1396) at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:495) at
org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:256) at
org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1158)
at
org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1042)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:974)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:888)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:632)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:605)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1079)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1054)
at
org.apache.hudi.index.hbase.TestSparkHoodieHBaseIndex.init(TestSparkHoodieHBaseIndex.java:129)
{code}
After Spark 4 support is merged, need to enable this test and make it working
with Spark 4.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
