[
https://issues.apache.org/jira/browse/KAFKA-5827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Behrang Saeedzadeh updated KAFKA-5827:
--------------------------------------
Description:
As far as I can see, Kafka connectors start exporting data of a topic from the
beginning of its partitions. We have a topic that contains a few million old
records that we don't need but we would like to start exporting new records
that are added to the topic.
Basically:
* When the connector is started for the first time and it does not have a
current offset stored, it should start consuming data from the end of topic
partitions
* When the connector is restarted and has a current offset for partitions
stored somewhere, it should start from those offsets
was:
As far as I can see, Kafka connectors start exporting data of a topic from its
beginning. We have a topic that contains a few million old records that we
don't need but we would like to start exporting new records that are added to
the topic.
Basically:
* When the connector is started for the first time and it does not have a
current offset stored, it should start consuming data from the end of topics
* When the connector is restarted and has a current offset stored somewhere, it
should start from that offset
> Allow configuring Kafka sink connectors to start processing records from the
> end of topics
> ------------------------------------------------------------------------------------------
>
> Key: KAFKA-5827
> URL: https://issues.apache.org/jira/browse/KAFKA-5827
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Behrang Saeedzadeh
>
> As far as I can see, Kafka connectors start exporting data of a topic from
> the beginning of its partitions. We have a topic that contains a few million
> old records that we don't need but we would like to start exporting new
> records that are added to the topic.
> Basically:
> * When the connector is started for the first time and it does not have a
> current offset stored, it should start consuming data from the end of topic
> partitions
> * When the connector is restarted and has a current offset for partitions
> stored somewhere, it should start from those offsets
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
