[
https://issues.apache.org/jira/browse/KAFKA-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643036#comment-14643036
]
Ewen Cheslack-Postava commented on KAFKA-2365:
----------------------------------------------
Thanks, auto assignment works fine, I appreciate the click savings.
On JDBC and HDFS, I haven't converted them over to use this patch in Kafka yet
(previously they were using the code in a separate repository under an
io.confluent package), but here are two prototypes:
https://github.com/confluentinc/copycat-jdbc and
https://github.com/confluentinc/copycat-hdfs These are incomplete, but have
enough basic functionality to work as POC, and may also be useful in evaluating
the core connector APIs for KAFKA-2366.
I think there are two things we should address before pulling any of that code
in. 1. We should probably discuss what the process is for pulling a connector
into Kafka core, especially when it means additional dependencies. This
includes how we manage releases. Should connectors go under contrib like the
hadoop-producer and hadoop-consumer? Should votes be done for specific
connectors since each one means taking on additional maintenance burden? 2.
Structurally, I'd like to make sure we sort out how all these artifacts will
show up in the builds. I mentioned this in KAFKA-2366, but before refactoring
any more code to fit into the build, it'd be nice to make sure we have the code
organization we want to stick with. (If you can't tell, I prefer to avoid
mucking with the gradle build as little as possible...)
> Copycat checklist
> -----------------
>
> Key: KAFKA-2365
> URL: https://issues.apache.org/jira/browse/KAFKA-2365
> Project: Kafka
> Issue Type: New Feature
> Components: copycat
> Reporter: Ewen Cheslack-Postava
> Assignee: Ewen Cheslack-Postava
> Labels: feature
> Fix For: 0.8.3
>
>
> This covers the development plan for
> [KIP-26|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=58851767].
> There are a number of features that can be developed in sequence to make
> incremental progress, and often in parallel:
> * Initial patch - connector API and core implementation
> * Runtime data API
> * Standalone CLI
> * REST API
> * Distributed copycat - CLI
> * Distributed copycat - coordinator
> * Distributed copycat - config storage
> * Distributed copycat - offset storage
> * Log/file connector (sample source/sink connector)
> * Elasticsearch sink connector (sample sink connector for full log -> Kafka
> -> Elasticsearch sample pipeline)
> * Copycat metrics
> * System tests (including connector tests)
> * Mirrormaker connector
> * Copycat documentation
> This is an initial list, but it might need refinement to allow for more
> incremental progress and may be missing features we find we want before the
> initial release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
