Re: Number of parallel connections for Elasticsearch Connector

2021-01-17 Thread Rex Fenley
Great, thanks! On Sun, Jan 17, 2021 at 6:24 PM Yangze Guo wrote: > Hi, Rex. > > > How many connections does the ES connector use to write to Elasticsearch? > I think the number is equal to your parallelism. Each subtask of an > Elasticsearch sink will have its own separate Bulk Processor as both

Re: Number of parallel connections for Elasticsearch Connector

2021-01-17 Thread Yangze Guo
Hi, Rex. > How many connections does the ES connector use to write to Elasticsearch? I think the number is equal to your parallelism. Each subtask of an Elasticsearch sink will have its own separate Bulk Processor as both the Client and the Bulk Processor are class private[1]. The subtasks will be

Re: Number of parallel connections for Elasticsearch Connector

2021-01-16 Thread Rex Fenley
I found the following, indicating that there is no concurrency for the Elasticsearch Connector https://github.com/apache/flink/blob/97bfd049951f8d52a2e0aed14265074c4255ead0/flink-connectors/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/Elastics