lamber-ken commented on a change in pull request #1317: [HUDI-605] Avoid
calculating the size of schema redundantly
URL: https://github.com/apache/incubator-hudi/pull/1317#discussion_r377734447
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/HoodieRecordSizeEstimator.java
##########
@@ -49,8 +48,9 @@ public long sizeEstimate(HoodieRecord<T> hoodieRecord) {
// note the sizes and differences. A correct estimation in such cases is
handled in
/** {@link ExternalSpillableMap} **/
long sizeOfRecord = ObjectSizeCalculator.getObjectSize(hoodieRecord);
- long sizeOfSchema = ObjectSizeCalculator.getObjectSize(schema);
- LOG.info("SizeOfRecord => " + sizeOfRecord + " SizeOfSchema => " +
sizeOfSchema);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("SizeOfRecord => " + sizeOfRecord + " SizeOfSchema => " +
sizeOfSchema);
+ }
Review comment:
More, HoodieRecordSizeEstimator is mainly used in `ExternalSpillableMap`,
`ExternalSpillableMap#put` has logged it.
```
if (shouldEstimatePayloadSize && estimatedPayloadSize == 0) {
// At first, use the sizeEstimate of a record being inserted into the
spillable map.
// Note, the converter may over estimate the size of a record in the JVM
this.estimatedPayloadSize = keySizeEstimator.sizeEstimate(key) +
valueSizeEstimator.sizeEstimate(value);
LOG.info("Estimated Payload size => " + estimatedPayloadSize);
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
