[
https://issues.apache.org/jira/browse/FLINK-17545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Lonergan updated FLINK-17545:
----------------------------------
Description:
See also FLINK-17544 for background to this scenario.
Synchronisation in JDBCUpsertOutputFormat allows resources leaks.
The close() method is synchronised with write() and flush() but not open.
So if open() is a slow call (which in my case it occasionally is) then it may
not be complete when close() is called.
In this situation the open() method will eventually proceed to open resources
such as database connections that will never be closed by the program.
So if we are +not+ going to synchronise open() with the other methods then an
alternative approach would be to have open() do a double check before returning.
ie Just before open() completes it would need to synchonise and then check if
close has already been called whilst open() was running.
If so then open() should proceed to undo any resources it has just opened.
One could perhaps argue that there's a problem with the state model if the
framework might call close before open has completed; but I can imagine why
this happens.
However, the must therefore deal with the consequential complexity of this deal
with the threading issues effectively.
was:
See also Flink-17544 for background to this scenario.
Synchronisation in JDBCUpsertOutputFormat allows resources leaks.
The close() method is synchronised with write() and flush() but not open.
So if open() is a slow call (which in my case it occasionally is) then it may
not be complete when close() is called.
In this situation the open() method will eventually proceed to open resources
such as database connections that will never be closed by the program.
So if we are +not+ going to synchronise open() with the other methods then an
alternative approach would be to have open() do a double check before returning.
ie Just before open() completes it would need to synchonise and then check if
close has already been called whilst open() was running.
If so then open() should proceed to undo any resources it has just opened.
One could perhaps argue that there's a problem with the state model if the
framework might call close before open has completed; but I can imagine why
this happens.
However, the must therefore deal with the consequential complexity of this deal
with the threading issues effectively.
> Resource leak JDBCUpsertOutputFormat
> ------------------------------------
>
> Key: FLINK-17545
> URL: https://issues.apache.org/jira/browse/FLINK-17545
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / JDBC
> Affects Versions: 1.10.0
> Reporter: John Lonergan
> Priority: Major
>
> See also FLINK-17544 for background to this scenario.
> Synchronisation in JDBCUpsertOutputFormat allows resources leaks.
> The close() method is synchronised with write() and flush() but not open.
> So if open() is a slow call (which in my case it occasionally is) then it may
> not be complete when close() is called.
> In this situation the open() method will eventually proceed to open resources
> such as database connections that will never be closed by the program.
> So if we are +not+ going to synchronise open() with the other methods then an
> alternative approach would be to have open() do a double check before
> returning.
> ie Just before open() completes it would need to synchonise and then check if
> close has already been called whilst open() was running.
> If so then open() should proceed to undo any resources it has just opened.
> One could perhaps argue that there's a problem with the state model if the
> framework might call close before open has completed; but I can imagine why
> this happens.
> However, the must therefore deal with the consequential complexity of this
> deal with the threading issues effectively.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
