Will do. Thank you for digging in and finding this!
On Mon, Jan 18, 2021 at 3:01 AM Dawid Wysakowicz
wrote:
> I've checked it again, with ES 7 this time (I missed that previously and
> checked with ES 6) and actually you're right it fails for value -1. I
> created a JIRA[1] to fix it. Until this
I've checked it again, with ES 7 this time (I missed that previously and
checked with ES 6) and actually you're right it fails for value -1. I
created a JIRA[1] to fix it. Until this is merged unfortunately, I can't
think of a way to actually disable it.
For a future reference. It would be really
Yes, use the same SQL and change '0's to '-1'. We received "Caused by:
java.lang.IllegalArgumentException: Could not parse value '-1' for key
'sink.bulk-flush.max-size'."
On Fri, Jan 15, 2021 at 6:04 AM Dawid Wysakowicz
wrote:
> Hi Rex,
>
> As I said in my previous email the documentation for
>
Hi Rex,
As I said in my previous email the documentation for
sink.bulk-flush.max-actions is wrong. You should be able to disable it
with -1. I've just checked it on the 1.11.2 tag and it seems to be
working just fine with:
CREATE TABLE esTable (
a BIGINT NOT NULL,
b TIME,
c STRING NO
Flink 1.11.2
CREATE TABLE sink_es (
...
) WITH (
'connector' = 'elasticsearch-7',
'hosts' = '${sys:proxyEnv.ELASTICSEARCH_HOSTS}',
'index' = '${sys:graph.flink.index_name}',
'format' = 'json',
'sink.bulk-flush.max-actions' = '0',
'sink.bulk-flush.max-size' = '0',
'sink.bulk-flush.interval' = '1s',
Hi,
First of all, what Flink versions are you using?
You are right it is a mistake in the documentation of the
sink.bulk-flush.max-actions. It should say: Can be set to |'-1'| to
disable it. I created a ticket[1] to track that. And as far as I can
tell and I quickly checked that it should work. A
Hello,
It doesn't seem like we can disable max actions and max size for
Elasticsearch connector.
Docs:
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connectors/elasticsearch.html#sink-bulk-flush-max-actions
sink.bulk-flush.max-actions optional 1000 Integer Maximum number
