ggershinsky commented on code in PR #3148:
URL: https://github.com/apache/parquet-java/pull/3148#discussion_r1949011211
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java:
##########
@@ -1146,12 +1244,60 @@ public void writeDataPageV2(
byte[] pageHeaderAAD,
SizeStatistics sizeStatistics)
throws IOException {
+ writeDataPageV2(
+ rowCount,
+ nullCount,
+ valueCount,
+ repetitionLevels,
+ definitionLevels,
+ dataEncoding,
+ compressedData,
+ true, /* compressed by default */
+ uncompressedDataSize,
+ statistics,
+ metadataBlockEncryptor,
+ pageHeaderAAD,
+ sizeStatistics);
+ }
+
+ /**
+ * Writes a single v2 data page
+ *
+ * @param rowCount count of rows
+ * @param nullCount count of nulls
+ * @param valueCount count of values
+ * @param repetitionLevels repetition level bytes
+ * @param definitionLevels definition level bytes
+ * @param dataEncoding encoding for data
+ * @param bytes data bytes
+ * @param compressed whether the data bytes is compressed
+ * @param uncompressedDataSize the size of uncompressed data
+ * @param statistics the statistics of the page
+ * @param metadataBlockEncryptor encryptor for block data
+ * @param pageHeaderAAD pageHeader AAD
+ * @param sizeStatistics size statistics for the page
+ * @throws IOException if any I/O error occurs during writing the file
+ */
+ public void writeDataPageV2(
+ int rowCount,
+ int nullCount,
+ int valueCount,
+ BytesInput repetitionLevels,
+ BytesInput definitionLevels,
+ Encoding dataEncoding,
+ BytesInput bytes,
+ boolean compressed,
+ int uncompressedDataSize,
+ Statistics<?> statistics,
+ BlockCipher.Encryptor metadataBlockEncryptor,
+ byte[] pageHeaderAAD,
+ SizeStatistics sizeStatistics)
+ throws IOException {
state = state.write();
int rlByteLength = toIntWithCheck(repetitionLevels.size(), "page
repetition levels");
int dlByteLength = toIntWithCheck(definitionLevels.size(), "page
definition levels");
- int compressedSize =
- toIntWithCheck(compressedData.size() + repetitionLevels.size() +
definitionLevels.size(), "page");
+ int compressedSize = toIntWithCheck(bytes.size() + repetitionLevels.size()
+ definitionLevels.size(), "page");
Review Comment:
The AAD suffix parameters are calculated in-memory, they are not stored in
the file.
The `compressedSize` does include the encryption IV/tag (28 bytes for GCM),
but this was already accounted for in the old `compressedData.size()`. I
presume the new `bytes.size()` is the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@parquet.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@parquet.apache.org
For additional commands, e-mail: issues-h...@parquet.apache.org
