Hi Rong, We have already tried to set the fetch size with no success. According to PG documentation we have to set both configuration parameters (i.e., auto-commit to false and limit fetch) to achieve our purpose.
Thanks, Konstantinos From: Rong Rong <walter...@gmail.com> Sent: Παρασκευή, 12 Απριλίου 2019 6:50 μμ To: Papadopoulos, Konstantinos <konstantinos.papadopou...@iriworldwide.com> Cc: user <user@flink.apache.org> Subject: Re: Flink JDBC: Disable auto-commit mode Hi Konstantinos, Seems like setting for auto commit is not directly possible in the current JDBCInputFormatBuilder. However there's a way to specify the fetch size [1] for your DB round-trip, doesn't that resolve your issue? Similarly in JDBCOutputFormat, a batching mode was also used to stash upload rows before flushing to DB. -- Rong [1] https://docs.oracle.com/cd/E18283_01/java.112/e16548/resltset.htm#insertedID4<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.oracle.com%2Fcd%2FE18283_01%2Fjava.112%2Fe16548%2Fresltset.htm%23insertedID4&data=02%7C01%7C%7C44d9fa279b674725483008d6bf5e7b3e%7C43728c2044474b27ac2e4bdabb3c0121%7C0%7C1%7C636906809861700178&sdata=XE%2FjZhh%2F8dcBe679%2FccdGq0rutlAiXHBHTj45eoXM70%3D&reserved=0> On Fri, Apr 12, 2019 at 6:23 AM Papadopoulos, Konstantinos <konstantinos.papadopou...@iriworldwide.com<mailto:konstantinos.papadopou...@iriworldwide.com>> wrote: Hi all, We are facing an issue when trying to integrate PostgreSQL with Flink JDBC. When you establish a connection to the PostgreSQL database, it is in auto-commit mode. It means that each SQL statement is treated as a transaction and is automatically committed, but this functionality results in unexpected behavior (e.g., out-of-memory errors) when executed for large result sets. In order to bypass such issues, we must disable the auto-commit mode. To do this, in a simple Java application, we call the setAutoCommit() method of the Connection object. So, my question is: How can we achieve this by using JDBCInputFormat of Flink? Thanks in advance, Konstantinos
