[
https://issues.apache.org/jira/browse/FLINK-5518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakub Havlik updated FLINK-5518:
--------------------------------
Description:
When developing a simple Flink applications reading ORC files it crashes with
NullPointerException when number of instances/executor threads is higher then
the number of files because it is trying to close a HadoopInputFormat which is
trying to close RecordReader which was not yet initialized as there is no file
for which it should have been opened. The issue is caused when
{code:java}
public void run(SourceContext<OUT> ctx) throws Exception {
try {
...
while (isRunning) {
format.open(splitIterator.next());
...
} finally {
format.close();
...
}
{code}
in file {{InputFormatSourceFunction.java}} which calls
{code:java}
public void close() throws IOException {
// enforce sequential close() calls
synchronized (CLOSE_MUTEX) {
this.recordReader.close();
}
}
{code}
from {{HadoopInputFormatBase.java}}.
As there is just this one implementation of the {{close()}} method it may be
enough just to add a null check for the {{this.recordReader}} in there.
was:
When developing a simple Flink applications reading ORC files it crashes with
NullPointerException when number of instances/executor threads is higher then
the number of files because it is trying to close a HadoopInputFormat which is
trying to close RecordReader which was not yet initialized as there is no file
for which it should have been opened. The issue is caused when
{code:java}
public void run(SourceContext<OUT> ctx) throws Exception {
try {
...
while (isRunning) {
format.open(splitIterator.next());
...
} finally {
format.close();
...
}
{code}
in file {{InputFormatSourceFunction.java}} which calls
{code:java}
public void close() throws IOException {
// enforce sequential close() calls
synchronized (CLOSE_MUTEX) {
this.recordReader.close();
}
}
{code}
from {{HadoopInputFormatBase.java}}.
As there is just this one implementation of the {{close()}} method it may be
enough just to add a null check for the {{this.recordReader}} in there.
> HadoopInputFormat throws NPE when close() is called before open()
> -----------------------------------------------------------------
>
> Key: FLINK-5518
> URL: https://issues.apache.org/jira/browse/FLINK-5518
> Project: Flink
> Issue Type: Bug
> Components: Batch Connectors and Input/Output Formats
> Affects Versions: 1.1.4
> Reporter: Jakub Havlik
> Labels: beginner, easyfix, newbie
>
> When developing a simple Flink applications reading ORC files it crashes with
> NullPointerException when number of instances/executor threads is higher then
> the number of files because it is trying to close a HadoopInputFormat which
> is trying to close RecordReader which was not yet initialized as there is no
> file for which it should have been opened. The issue is caused when
> {code:java}
> public void run(SourceContext<OUT> ctx) throws Exception {
> try {
> ...
> while (isRunning) {
> format.open(splitIterator.next());
> ...
> } finally {
> format.close();
> ...
> }
> {code}
> in file {{InputFormatSourceFunction.java}} which calls
> {code:java}
> public void close() throws IOException {
> // enforce sequential close() calls
> synchronized (CLOSE_MUTEX) {
> this.recordReader.close();
> }
> }
> {code}
> from {{HadoopInputFormatBase.java}}.
> As there is just this one implementation of the {{close()}} method it may be
> enough just to add a null check for the {{this.recordReader}} in there.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
