[DISCUSS] FLINK-33857 connector-opensearch|elasticsearch: expose flush

2023-12-21 Thread Peter Fischer
Hi! We encountered a bug in elasticsearch recently that prevents us from using the flink connectors for opensearch|elaticsearch: size estimations for bulk requests ignore potentially large parameter values, see [1]. As a workaround, we patched the connector to expose the flush method of the underl

[FLINK-32028] connectors/elasticsearch: error handling

2023-12-05 Thread Peter Fischer
Hi! We're using flink and its elasticsearch (ES) sink to process content changes at wikimedia. The connector uses ES' bulk API but is rather strict when it comes to interpreting a response: It fails if a single action of the bulk has failed. This behavior is not configurable at the moment so I wou