[
https://issues.apache.org/jira/browse/FLINK-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-18826:
----------------------------
Description:
Currently, it is not possible to emit the results of a group by query into
Kafka. We can provide a format (e.g. {{upsert-json}}??) to encode changelogs to
make it possible to write into Kafka. Another idea is provide an
{{update-mode=upsert}} property, in this way, we can support upsert behavior
for all the append-only formats (avro, avro-confluent, json, csv) without
changing the implementation of the formats. The update-mode came from kafka
connector [1] before 1.10, but was dropped after that.
[1]:
was:Currently, it is not possible to emit the results of a group by query
into Kafka. We can provide a format (e.g. {{upsert-json}}??) to encode
changelogs to make it possible to write into Kafka.
> Support to emit and encode upsert messages to Kafka
> ---------------------------------------------------
>
> Key: FLINK-18826
> URL: https://issues.apache.org/jira/browse/FLINK-18826
> Project: Flink
> Issue Type: Sub-task
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
> Reporter: Jark Wu
> Assignee: Shengkai Fang
> Priority: Major
>
> Currently, it is not possible to emit the results of a group by query into
> Kafka. We can provide a format (e.g. {{upsert-json}}??) to encode changelogs
> to make it possible to write into Kafka. Another idea is provide an
> {{update-mode=upsert}} property, in this way, we can support upsert behavior
> for all the append-only formats (avro, avro-confluent, json, csv) without
> changing the implementation of the formats. The update-mode came from kafka
> connector [1] before 1.10, but was dropped after that.
> [1]:
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
