[
https://issues.apache.org/jira/browse/SPARK-57958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-57958:
------------------------------------
Assignee: Hyukjin Kwon
> Close the ORC Reader in OrcSourceSuite to fix leaked file streams
> -----------------------------------------------------------------
>
> Key: SPARK-57958
> URL: https://issues.apache.org/jira/browse/SPARK-57958
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
>
> OrcSuite.testBloomFilterCreation and testSelectiveDictionaryEncoding create
> an ORC Reader via OrcFile.createReader(...) but only close the derived
> RecordReader in the finally block, never the Reader itself. The Reader holds
> the file stream opened in ReaderImpl.extractFileTail, so it leaks.
> SharedSparkSession.afterEach asserts no open streams
> (DebugFilesystem.assertNoOpenStreams) and, when the leaked stream is not
> closed within the 10s window, aborts the whole suite: 'There are 1 possibly
> leaked file streams'. This flakes OrcSourceV1Suite/OrcSourceV2Suite, e.g. on
> build_java21 (branch-4.0). Fix: also close the Reader in the finally block.
> Test-only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
