Hey folks, I'm trying to use flink to write high throughput incoming data to a SQL db using the JDBC Connector as described here:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/jdbc.html However, after enabling this, my data consumption rate slowed down to a crawl. After doing some digging, it seemed like the number of connections to the db was very low, almost 1 connection per worker (there are 9 workers). Which seemed like the culprit if Flink was blocking and waiting for a single db connection to do all this work? Is there a way to tell the JDBC connector to use more db connections? Or do I need to specifically increase the parallelism of the connector to beyond the default? And if I need to increase the parallelism, currently my setup is having 9 workers, 1 task slot for each (this cluster is dedicated to one job). That means I have to increase the number of task slots before increasing parallelism, right? My flink version is 1.10.1 and my jdbc connection is flink-connector-jdbc_2.11:1.11.0. Thanks! Li
