yihua commented on code in PR #12772: URL: https://github.com/apache/hudi/pull/12772#discussion_r2074258693
########## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/index/hbase/TestSparkHoodieHBaseIndex.java: ########## @@ -102,6 +102,7 @@ * (see one problem here : https://issues.apache.org/jira/browse/HBASE-15835). Hence, the need to use * {@link MethodOrderer.Alphanumeric} to make sure the tests run in order. Please alter the order of tests running carefully. */ +@Disabled Review Comment: Could we add a condition to run this test class on Spark 3.5 and below only? ########## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/commit/TestCopyOnWriteActionExecutor.java: ########## @@ -575,4 +574,24 @@ public void testPartitionMetafileFormat(boolean partitionMetafileUseBaseFormat) assertTrue(partitionMetadata.readPartitionCreatedCommitTime().get().equals(instantTime)); } + // methods below were copied from [[TestBulkInsertInternalPartitioner]] + public static JavaRDD<HoodieRecord> generateTestRecordsForBulkInsert(JavaSparkContext jsc) { + HoodieTestDataGenerator dataGenerator = new HoodieTestDataGenerator(); + // RDD partition 1 + List<HoodieRecord> records1 = dataGenerator.generateInserts("0", 100); + // RDD partition 2 + List<HoodieRecord> records2 = dataGenerator.generateInserts("0", 150); + return jsc.parallelize(records1, 1).union(jsc.parallelize(records2, 1)); + } + + public static JavaRDD<HoodieRecord> generateTestRecordsForBulkInsert(JavaSparkContext jsc, int count) { + HoodieTestDataGenerator dataGenerator = new HoodieTestDataGenerator(); + List<HoodieRecord> records = dataGenerator.generateInserts("0", count); + return jsc.parallelize(records, 1); + } + + public static Map<String, Long> generateExpectedPartitionNumRecords(JavaRDD<HoodieRecord> records) { + return records.map(record -> record.getPartitionPath()).countByValue(); + } + Review Comment: nit: remove empty line ########## hudi-common/src/main/java/org/apache/hudi/avro/HoodieBloomFilterWriteSupport.java: ########## @@ -55,15 +55,19 @@ public HoodieBloomFilterWriteSupport(BloomFilter bloomFilter) { public void addKey(T recordKey) { bloomFilter.add(getUTF8Bytes(recordKey)); - if (minRecordKey == null || minRecordKey.compareTo(recordKey) > 0) { + if (minRecordKey == null || compareRecordKey(minRecordKey, recordKey) > 0) { Review Comment: Similar here. If `HoodieUTF8StringKey` is passed in which has overridden `compareTo` implementation, there is no need to change this class. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
