[jira] [Commented] (KAFKA-2887) TopicMetadataRequest creates topic if it does not exist

2015-11-24 Thread Yi Pan (Data Infrastructure) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025854#comment-15025854 ] Yi Pan (Data Infrastructure) commented on KAFKA-2887: -

[jira] [Commented] (KAFKA-2804) Create / Update changelog topics upon state store initialization

2015-11-24 Thread Yi Pan (Data Infrastructure) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025815#comment-15025815 ] Yi Pan (Data Infrastructure) commented on KAFKA-2804: - [~guoz

[jira] [Commented] (KAFKA-2887) TopicMetadataRequest creates topic if it does not exist

2015-11-24 Thread Yi Pan (Data Infrastructure) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025804#comment-15025804 ] Yi Pan (Data Infrastructure) commented on KAFKA-2887: - Agree

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-27 Thread Yi Pan
tas, which are in any case getting implemented. > > -Jay > > On Sun, Jul 26, 2015 at 11:31 PM, Yi Pan wrote: > > > Hi, Jay and all, > > > > Thanks for all your quick responses. I tried to summarize my thoughts > here: > > > > - ConsumerRecord a

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-27 Thread Yi Pan
Hi, Neha, {quote} We do hope to include a DSL since that is the most natural way of expressing stream processing operations on top of the processor client. The DSL layer should be equivalent to that provided by Spark streaming or Flink in terms of expressiveness though there will be differences in

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-27 Thread Yi Pan
Hi, Aditya, {quote} - The KIP states that cmd line tools will be provided to deploy as a separate service. Is the proposed scope limited to providing a library with which makes it possible build stream-processing-as- a-service or provide such a service within Kafka itself? {quote} There has alrea

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-26 Thread Yi Pan
t-class ingestion framework > > So the case where you have data coming from many systems including many > Kafka clusters is just about how easy/hard it is to use copycat with the > transformer api in the same program. I think this is something we should > work out as part of the proto

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-24 Thread Yi Pan
Hi, Guozhang, Thanks for starting this. I took a quick look and had the following thoughts to share: - In the proposed KafkaProcessor API, there is no interface like Collector that allows users to send messages to. Why is that? Is the idea to initialize the producer once and re-use it in the proc

Re: Review Request 34607: Fix SAMZA-686: adding test for max changelog partition computation

2015-05-23 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34607/#review85083 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On May 22