gszadovszky commented on code in PR #3126:
URL: https://github.com/apache/parquet-java/pull/3126#discussion_r1921884885
##########
parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java:
##########
@@ -2382,8 +2382,14 @@ public static SizeStatistics
toParquetSizeStatistics(org.apache.parquet.column.s
formatStats.setUnencoded_byte_array_data_bytes(
stats.getUnencodedByteArrayDataBytes().get());
}
-
formatStats.setRepetition_level_histogram(stats.getRepetitionLevelHistogram());
-
formatStats.setDefinition_level_histogram(stats.getDefinitionLevelHistogram());
+ List<Long> repLevelHistogram = stats.getRepetitionLevelHistogram();
+ if (repLevelHistogram != null && !repLevelHistogram.isEmpty()) {
+ formatStats.setRepetition_level_histogram(repLevelHistogram);
+ }
+ List<Long> defLevelHistogram = stats.getDefinitionLevelHistogram();
+ if (defLevelHistogram != null && !defLevelHistogram.isEmpty()) {
+ formatStats.setDefinition_level_histogram(defLevelHistogram);
+ }
Review Comment:
Same question as before: Was it an actual defect?
##########
parquet-column/src/main/java/org/apache/parquet/column/statistics/SizeStatistics.java:
##########
@@ -136,8 +136,10 @@ public SizeStatistics(
List<Long> definitionLevelHistogram) {
this.type = type;
this.unencodedByteArrayDataBytes = unencodedByteArrayDataBytes;
- this.repetitionLevelHistogram = repetitionLevelHistogram;
- this.definitionLevelHistogram = definitionLevelHistogram;
+ this.repetitionLevelHistogram =
+ repetitionLevelHistogram == null ? Collections.emptyList() :
repetitionLevelHistogram;
+ this.definitionLevelHistogram =
+ definitionLevelHistogram == null ? Collections.emptyList() :
definitionLevelHistogram;
Review Comment:
Was it an actual defect that may cause issues? Shouldn't we backport this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@parquet.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@parquet.apache.org
For additional commands, e-mail: issues-h...@parquet.apache.org
