[
https://issues.apache.org/jira/browse/IGNITE-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15071500#comment-15071500
]
Denis Magda commented on IGNITE-2016:
-------------------------------------
Roman,
According to {{SinkTask.put(...)}} API
http://docs.confluent.io/2.0.0/connect/javadocs/org/apache/kafka/connect/sink/SinkTask.html#put(java.util.Collection)
There is no need to buffer the data waiting for {{SinkTask.flush()}} to be
called. It's ok and in my opinion preferable to start sending the data
asynchronously.
I got your idea and concerns regarding the case when something going wrong
during the streaming and we want to let Kafka know to continue streaming from
some offset. However to overcome this issue we still can rely on
{{IgniteDataStreamer}}. Just call {{IgniteDataStreamer.flush()}} in the
implementation of your sink's flush method. This will ensure that the streamer
has already flushed all the data properly at the time when implementation of
your {{flush}} method returns.
Makes sense?
> Update KafkaStreamer to fit new features introduced in Kafka 0.9
> ----------------------------------------------------------------
>
> Key: IGNITE-2016
> URL: https://issues.apache.org/jira/browse/IGNITE-2016
> Project: Ignite
> Issue Type: New Feature
> Components: streaming
> Reporter: Roman Shtykh
> Assignee: Roman Shtykh
>
> Particularly,
> - new consumer
> - Kafka Connect (Copycat)
> http://www.confluent.io/blog/apache-kafka-0.9-is-released
> This can be a a different integration task or a complete re-write of the
> current implementation, considering the fact that Kafka Connect is a new
> standard way for "large-scale, real-time data import and export for Kafka."
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
