Re: jdbc connector configuration

2021-10-18 Thread Arvid Heise
K8s should not restart a finished job. Are you seeing this? How did you configure the job? On Wed, Oct 13, 2021 at 7:39 AM Qihua Yang wrote: > Hi, > > If I configure batch mode, application will stop after the job is > complete, right? Then k8s will restart the pod and rerun the job. That is > n

Re: jdbc connector configuration

2021-10-12 Thread Qihua Yang
Hi, If I configure batch mode, application will stop after the job is complete, right? Then k8s will restart the pod and rerun the job. That is not what we want. Thanks, Qihua On Tue, Oct 12, 2021 at 7:27 PM Caizhi Weng wrote: > Hi! > > It seems that you want to run a batch job instead of a st

Re: jdbc connector configuration

2021-10-12 Thread Caizhi Weng
Hi! It seems that you want to run a batch job instead of a streaming job. Call EnvironmentSettings.newInstance().inBatchMode().build() to create your environment settings for a batch job. Qihua Yang 于2021年10月13日周三 上午5:50写道: > Hi, > > Sorry for asking again. I plan to use JDBC connector to scan

Re: jdbc connector configuration

2021-10-12 Thread Qihua Yang
Hi, Sorry for asking again. I plan to use JDBC connector to scan a database. How do I know if it is done? Are there any metrics I can track? We want to monitor the progress, stop flink application when it is done. Thanks, Qihua On Fri, Oct 8, 2021 at 10:07 AM Qihua Yang wrote: > It is pretty c

Re: jdbc connector configuration

2021-10-08 Thread Qihua Yang
It is pretty clear. Thanks Caizhi! On Thu, Oct 7, 2021 at 7:27 PM Caizhi Weng wrote: > Hi! > > These configurations are not required to merely read from a database. They > are here to accelerate the reads by allowing sources to read data in > parallel. > > This optimization works by dividing the

Re: jdbc connector configuration

2021-10-07 Thread Caizhi Weng
Hi! These configurations are not required to merely read from a database. They are here to accelerate the reads by allowing sources to read data in parallel. This optimization works by dividing the data into several (scan.partition.num) partitions and each partition will be read by a task slot (n

jdbc connector configuration

2021-10-06 Thread Qihua Yang
Hi, I am trying to read data from database with JDBC driver. From [1], I have to config below parameters. I am not quite sure if I understand it correctly. lower-bound is smallest value of the first partition, upper-bound is largest value of the last partition. For example, if the db table has 100