[
https://issues.apache.org/jira/browse/KAFKA-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136869#comment-15136869
]
Artyom commented on KAFKA-2375:
-------------------------------
Hi,
So me and my Team worked on this and implemented this connector, but using the
Elastic Search JAVA APIs. We didn't use just the plain REST interface for
performance concerns and it was much cleaner to implement throttling for the
indexing using the standard ES Bulk APIs and wrapping everything in a retry on
fail. This seems to work quite well and is fairly simple. We are interested in
sharing the connector, but not sure if there is interest in such an
implementation.
Artyom
> Implement elasticsearch Copycat sink connector
> ----------------------------------------------
>
> Key: KAFKA-2375
> URL: https://issues.apache.org/jira/browse/KAFKA-2375
> Project: Kafka
> Issue Type: Sub-task
> Components: copycat
> Reporter: Ewen Cheslack-Postava
> Assignee: Liquan Pei
>
> Implement an elasticsearch sink connector for Copycat. This should send
> records to elasticsearch with unique document IDs, given appropriate configs
> to extract IDs from input records.
> The motivation here is to provide a good end-to-end example with built-in
> connectors that require minimal dependencies. Because Elasticsearch has a
> very simple REST API, an elasticsearch connector shouldn't require any extra
> dependencies and logs -> Elasticsearch (in combination with KAFKA-2374)
> provides a compelling out-of-the-box Copycat use case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
