[
https://issues.apache.org/jira/browse/HIVE-27408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sourabh Badhya updated HIVE-27408:
----------------------------------
Description:
ParquetRecordWriterWrapper while closing the writer tries to collect the stats
by creating a reader (opening the file). But it never closes the reader (never
closes the file). This can leave the file open hence consuming memory and
associated file handle resources.
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java#L143-L155]
was:
ParquetRecordWriterWrapper while closing the writer tries to collect the stats
by opening a reader. But it never closes the reader. This can leave the file
open hence consuming memory and associated file handle resources.
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java#L143-L155
> Parquet file opened for reading stats is never closed
> -----------------------------------------------------
>
> Key: HIVE-27408
> URL: https://issues.apache.org/jira/browse/HIVE-27408
> Project: Hive
> Issue Type: Bug
> Reporter: Sourabh Badhya
> Assignee: Sourabh Badhya
> Priority: Major
> Labels: pull-request-available
>
> ParquetRecordWriterWrapper while closing the writer tries to collect the
> stats by creating a reader (opening the file). But it never closes the reader
> (never closes the file). This can leave the file open hence consuming memory
> and associated file handle resources.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java#L143-L155]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
