I found the following, indicating that there is no concurrency for the Elasticsearch Connector https://github.com/apache/flink/blob/97bfd049951f8d52a2e0aed14265074c4255ead0/flink-connectors/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBase.java#L382
Does each subtask of an Elasticsearch sink have it's own separate Bulk Processor to allow for parallel bulk writes? Thanks! On Sat, Jan 16, 2021 at 10:33 AM Rex Fenley <r...@remind101.com> wrote: > Hello, > > How many connections does the ES connector use to write to Elasticsearch? > We have a single machine with 16 vCPUs and parallelism of 4 running our > job, with -p 4 I'd expect there to be 4 parallel bulk request writers / > connections to Elasticsearch. Is there a place in the code to confirm this? > > Thanks! > > -- > > Rex Fenley | Software Engineer - Mobile and Backend > > > Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> | > FOLLOW US <https://twitter.com/remindhq> | LIKE US > <https://www.facebook.com/remindhq> > -- Rex Fenley | Software Engineer - Mobile and Backend Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> | FOLLOW US <https://twitter.com/remindhq> | LIKE US <https://www.facebook.com/remindhq>
