s on the Elasticsearch you're using.
De: ApoorvK
Enviat el: dilluns, 10 de febrer de 2020 15:33
Per a: user@flink.apache.org
Tema: Flink Elasticsearch upsert document in ES
Team,
Presently I have added elasticsearch as a sink to a stream and inserting the
json
Team,
Presently I have added elasticsearch as a sink to a stream and inserting the
json data, the problem is when I restore the application in case of crash it
reprocess the data in between (meanwhile a backend application updates the
document in ES) and flink reinsert the document in ES and all up
I have tried by providing opType to elasticsearch builder, I am getting an
error message "document already exists" on my console, but it still updates
the value in elasticsearch
val jsonString = write(record)
val rqst: IndexRequest = Requests.indexRequest
.index(parameter.get("esIndexName"))
.
Hi ApoorvK,
Elasticsearch supports "create" mode while indexing. By default indexing
will overwrite documents with a the same ID, but you can tell ES to refuse
overwriting. See op_type in
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#docs-index-api-query-params
.
Team,
Presently I have added elasticsearch as a sink to a stream and inserting the
json data, the problem is when I restore the application in case of crash it
reprocess the data in between (meanwhile a backend application updates the
document in ES) and flink reinsert the document in ES and all u
