cshuo commented on code in PR #13408:
URL: https://github.com/apache/hudi/pull/13408#discussion_r2139092656
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -183,6 +187,14 @@ public Option<Predicate> getKeyFilterOpt() {
return keyFilterOpt;
}
+ public SizeEstimator<BufferedRecord<T>> getRecordSizeEstimator() {
+ return new HoodieRecordSizeEstimator<>(schemaHandler.getRequiredSchema());
+ }
+
+ public CustomSerializer<BufferedRecord<T>> getRecordSerializer() {
+ return new DefaultSerializer<>();
Review Comment:
Btw, local benchmark results for
`BufferedRecordSerializer(AvroRecordSerializer)` with `DefualtSerializer`:
~~Legacy default: 1801s~~
Legacy default: 1268s
Legacy default: 1217s
Legacy default: 1204s
Legacy default: 1210s
Avg: 1224
~~New: 769s~~
New: 640s
New: 646s
New: 656s
New: 643s
Avg: 646
The Ser/De throughput of new serializer is about 2x of before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
