[GitHub] kafka pull request: MINOR: Replace deprecated property in MirrorMa...

GitHub user sasakitoa opened a pull request: https://github.com/apache/kafka/pull/733 MINOR: Replace deprecated property in MirrorMaker MirrorMaker uses deprecated property "block.on.buffer.full" as default. Replace this with "max.block.ms" which is recommended. You can merge t

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

Hi Adi, Please see inline comments. And thanks for correcting my typos :) > On Jan 5, 2016, at 3:37 AM, Aditya Auradkar > wrote: > > Hey Becket/Anna - > > I have a few comments about the KIP. > > 1. (Minor) Can we rename the KIP? It's currently "Add CreateTime and > LogAppendTime etc..". Thi

[jira] [Assigned] (KAFKA-3063) Transient exist(1) in unknown test cases

[ https://issues.apache.org/jira/browse/KAFKA-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reassigned KAFKA-3063: -- Assignee: Ismael Juma > Transient exist(1) in unknown test cases >

[jira] [Commented] (KAFKA-3063) Transient exist(1) in unknown test cases

[ https://issues.apache.org/jira/browse/KAFKA-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082825#comment-15082825 ] Ismael Juma commented on KAFKA-3063: I started looking into this just before the holid

[jira] [Updated] (KAFKA-3058) remove the usage of deprecated config properties

[ https://issues.apache.org/jira/browse/KAFKA-3058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konrad Kalita updated KAFKA-3058: - Status: Patch Available (was: Open) > remove the usage of deprecated config properties >

[jira] [Created] (KAFKA-3065) Prevent unbounded growth of RecordAccumulator#batches in Kafka producer

Rajini Sivaram created KAFKA-3065: - Summary: Prevent unbounded growth of RecordAccumulator#batches in Kafka producer Key: KAFKA-3065 URL: https://issues.apache.org/jira/browse/KAFKA-3065 Project: Kafk

[GitHub] kafka pull request: MINOR: Update `config/producer.properties` to ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/710 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] kafka pull request: KAFKA-2934: Offset storage file configuration ...

GitHub user ZoneMayor opened a pull request: https://github.com/apache/kafka/pull/734 KAFKA-2934: Offset storage file configuration in Connect standalone mode is not included in StandaloneConfig Added offsetBackingStore config to StandaloneConfig and DistributedConfig; Added con

[jira] [Commented] (KAFKA-2934) Offset storage file configuration in Connect standalone mode is not included in StandaloneConfig

[ https://issues.apache.org/jira/browse/KAFKA-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083117#comment-15083117 ] ASF GitHub Bot commented on KAFKA-2934: --- GitHub user ZoneMayor opened a pull request

Build failed in Jenkins: kafka-trunk-jdk7 #936

See Changes: [cshapi] MINOR: Update `config/producer.properties` to have new producer -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu3 (U

[jira] [Commented] (KAFKA-3050) Space in the value for "host.name" causes "Unresolved address"

[ https://issues.apache.org/jira/browse/KAFKA-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083195#comment-15083195 ] Navin Markandeya commented on KAFKA-3050: - [~ijuma] Yes. > Space in the value for

[jira] [Updated] (KAFKA-3050) Space in the value for "host.name" causes "Unresolved address"

[ https://issues.apache.org/jira/browse/KAFKA-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3050: --- Labels: newbie (was: ) > Space in the value for "host.name" causes "Unresolved address" > ---

Reg. Kafka - Socket Communication from external system

Hi, Please let us know whether KAFKA will support socket based communication with external GPS device. GPS Device will send GPS packet to the KAFKA IP and Port. Thanks in Advance Thanks & Regards, Prakash MN

Re: Reg. Kafka - Socket Communication from external system

Kafka requires a specific protocol and it is unlikely that GPS devices will support this protocol out of the box. The usual solution is to create a service that accepts GPS packets using the protocol that GPS devices support, converts it to Kafka records and produce them to a Kafka broker. Gwen

[jira] [Commented] (KAFKA-2426) A Kafka node tries to connect to itself through its advertised hostname

[ https://issues.apache.org/jira/browse/KAFKA-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083324#comment-15083324 ] Matt Hughes commented on KAFKA-2426: That workaround only works if you have a hostname

[jira] [Commented] (KAFKA-2426) A Kafka node tries to connect to itself through its advertised hostname

[ https://issues.apache.org/jira/browse/KAFKA-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083325#comment-15083325 ] Matt Hughes commented on KAFKA-2426: Elasticsearch had a similar issue and created two

Build failed in Jenkins: kafka-trunk-jdk8 #267

See Changes: [cshapi] MINOR: Update `config/producer.properties` to have new producer -- [...truncated 2834 lines...] kafka.integration.SaslSslTopicMetadataTest > testAutoCreateTopicWithCollision

[jira] [Commented] (KAFKA-3064) Improve resync method to waste less time and data transfer

[ https://issues.apache.org/jira/browse/KAFKA-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083379#comment-15083379 ] Michael Graff commented on KAFKA-3064: -- We have a 65 GB per partition retention polic

[jira] [Updated] (KAFKA-3064) Improve resync method to waste less time and data transfer

[ https://issues.apache.org/jira/browse/KAFKA-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Graff updated KAFKA-3064: - Description: We have several topics which are large (65 GB per partition) with 12 partitions. Da

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Jun/Allen - Did we ever actually agree on whether we should evolve the TMR to include rack info or not? I don't feel strongly about it but I if it's the right thing to do we should probably do it in this KIP (can be a separate patch).. it isn't a large change. Aditya On Sat, Dec 26, 2015 at 3:01

[jira] [Updated] (KAFKA-3062) Read from kafka replication to get data likely Version based

[ https://issues.apache.org/jira/browse/KAFKA-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xingang updated KAFKA-3062: --- Description: Since Kafka require all the reading happen in the leader for the consistency. If there could be

[jira] [Updated] (KAFKA-3062) Read from kafka replication to get data likely Version based

[ https://issues.apache.org/jira/browse/KAFKA-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xingang updated KAFKA-3062: --- Description: Since Kafka require all the reading happen in the leader for the consistency. in case, for large

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

+1, the KIP looks good to me now. Guozhang On Tue, Jan 5, 2016 at 1:47 AM, Becket Qin wrote: > Hi Adi, > > Please see inline comments. And thanks for correcting my typos :) > > > On Jan 5, 2016, at 3:37 AM, Aditya Auradkar > wrote: > > > > Hey Becket/Anna - > > > > I have a few comments about

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

+1 On Tue, Jan 5, 2016 at 11:12 AM, Guozhang Wang wrote: > +1, the KIP looks good to me now. > > Guozhang > > On Tue, Jan 5, 2016 at 1:47 AM, Becket Qin wrote: > > > Hi Adi, > > > > Please see inline comments. And thanks for correcting my typos :) > > > > > On Jan 5, 2016, at 3:37 AM, Aditya Au

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

Jiangjie, Thanks for the updated KIP. Overall, a +1 on the proposal. A few minor comments on the KIP. KIP-32: 50. 6.c says "The log rolling has to depend on the earliest timestamp", which is inconsistent with KIP-33. 51. 8.b "If the time difference threshold is set to 0. The timestamp in the mes

Re: KIP-41: KafkaConsumer Max Records

I've updated the KIP with some implementation details. I also added more discussion on the heartbeat() alternative. The short answer for why we rejected this API is that it doesn't seem to work well with offset commits. This would tend to make correct usage complicated and difficult to explain. Add

[jira] [Created] (KAFKA-3066) Add Demo Examples for Kafka Streams

Guozhang Wang created KAFKA-3066: Summary: Add Demo Examples for Kafka Streams Key: KAFKA-3066 URL: https://issues.apache.org/jira/browse/KAFKA-3066 Project: Kafka Issue Type: Sub-task

[GitHub] kafka pull request: KAFKA-3010; Include error in log when ack 0

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/709 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Commented] (KAFKA-3010) include error code when logging an error when ack = 0

[ https://issues.apache.org/jira/browse/KAFKA-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083641#comment-15083641 ] ASF GitHub Bot commented on KAFKA-3010: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-3010) include error code when logging an error when ack = 0

[ https://issues.apache.org/jira/browse/KAFKA-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3010: --- Resolution: Fixed Fix Version/s: 0.9.0.1 Status: Resolved (was: Patch Available) Issue reso

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

+1 on the KIP. Thanks for the hard work, Becket! On Tue, Jan 5, 2016 at 11:24 AM, Jun Rao wrote: > Jiangjie, > > Thanks for the updated KIP. Overall, a +1 on the proposal. A few minor > comments on the KIP. > > KIP-32: > 50. 6.c says "The log rolling has to depend on the earliest timestamp", > w

[GitHub] kafka pull request: KAFKA-3045; ZkNodeChangeNotificationListener s...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/731 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Resolved] (KAFKA-3045) ZkNodeChangeNotificationListener shouldn't log interrupted exception as error

[ https://issues.apache.org/jira/browse/KAFKA-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-3045. Resolution: Fixed Issue resolved by pull request 731 [https://github.com/apache/kafka/pull/731] > ZkNodeCha

[jira] [Commented] (KAFKA-3045) ZkNodeChangeNotificationListener shouldn't log interrupted exception as error

[ https://issues.apache.org/jira/browse/KAFKA-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083691#comment-15083691 ] ASF GitHub Bot commented on KAFKA-3045: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-3058) remove the usage of deprecated config properties

[ https://issues.apache.org/jira/browse/KAFKA-3058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3058: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 732 [https:/

[jira] [Commented] (KAFKA-3058) remove the usage of deprecated config properties

[ https://issues.apache.org/jira/browse/KAFKA-3058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083695#comment-15083695 ] ASF GitHub Bot commented on KAFKA-3058: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-3058: remove the usage of deprecated con...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/732 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Updated] (KAFKA-2937) Topics marked for delete in Zookeeper may become undeletable

[ https://issues.apache.org/jira/browse/KAFKA-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2937: --- Status: Patch Available (was: Open) There's a PR so updating the status. > Topics marked for delete

ZkSecurityMigrator incorrectly applies ACLs to entire ZooKeeper tree?

Hi, I'm running through some 0.8.2 to 0.9.0 upgrade testing that involves moving to a secured cluster - While running the zookeeper-security-migration.sh script, I noticed that it modifies ACLs for non-Kafka specific znodes/trees also. Looking at the code it seems like the intention is to only

Jenkins build is back to normal : kafka-trunk-jdk8 #268

See

[jira] [Commented] (KAFKA-3059) ConsumerGroupCommand should allow resetting offsets for consumer groups

[ https://issues.apache.org/jira/browse/KAFKA-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083922#comment-15083922 ] Jason Gustafson commented on KAFKA-3059: One problem is defining the behavior of t

[jira] [Created] (KAFKA-3067) Fix producer API documentation for case when RequiredAcks > 1

Sergiy Zuban created KAFKA-3067: --- Summary: Fix producer API documentation for case when RequiredAcks > 1 Key: KAFKA-3067 URL: https://issues.apache.org/jira/browse/KAFKA-3067 Project: Kafka Is

Build failed in Jenkins: kafka-trunk-jdk7 #937

See Changes: [junrao] KAFKA-3010; Include error in log when ack 0 [junrao] KAFKA-3045; ZkNodeChangeNotificationListener shouldn't log [junrao] KAFKA-3058; remove the usage of deprecated config properties -

[jira] [Commented] (KAFKA-3059) ConsumerGroupCommand should allow resetting offsets for consumer groups

[ https://issues.apache.org/jira/browse/KAFKA-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083935#comment-15083935 ] Gwen Shapira commented on KAFKA-3059: - One thing: If we go the route of "ask the users

[jira] [Work started] (KAFKA-3038) Speeding up partition reassignment after broker failure

[ https://issues.apache.org/jira/browse/KAFKA-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3038 started by Eno Thereska. --- > Speeding up partition reassignment after broker failure > ---

[jira] [Work started] (KAFKA-3013) Display the topic-partition in the exception message for expired batches in recordAccumulator

[ https://issues.apache.org/jira/browse/KAFKA-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3013 started by Mayuresh Gharat. -- > Display the topic-partition in the exception message for expired batches in

[jira] [Updated] (KAFKA-3013) Display the topic-partition in the exception message for expired batches in recordAccumulator

[ https://issues.apache.org/jira/browse/KAFKA-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-3013: --- Status: Patch Available (was: In Progress) > Display the topic-partition in the exception mes

[jira] [Work started] (KAFKA-1911) Log deletion on stopping replicas should be async

[ https://issues.apache.org/jira/browse/KAFKA-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-1911 started by Mayuresh Gharat. -- > Log deletion on stopping replicas should be async > ---

[jira] [Updated] (KAFKA-1911) Log deletion on stopping replicas should be async

[ https://issues.apache.org/jira/browse/KAFKA-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-1911: --- Status: Patch Available (was: In Progress) > Log deletion on stopping replicas should be asyn

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

+1. Great work Becket. Aditya On Tue, Jan 5, 2016 at 11:47 AM, Neha Narkhede wrote: > +1 on the KIP. Thanks for the hard work, Becket! > > On Tue, Jan 5, 2016 at 11:24 AM, Jun Rao wrote: > > > Jiangjie, > > > > Thanks for the updated KIP. Overall, a +1 on the proposal. A few minor > > comments

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Jun and I had a chance to discuss it in a meeting and it is agreed to change the TMR in a different patch. I can change the KIP to include rack in TMR. The essential change is to add rack into class BrokerEndPoint and make TMR version aware. On Tue, Jan 5, 2016 at 10:21 AM, Aditya Auradkar < aa

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

I'm +1 on the design, but this proposal doesn't look like it is quite finished. We are trying to make the KIPs really be a record of design decisions we made and why. We should be able to give the document to advanced Kafka users and have them walk away with a good idea of what we are changing and

[GitHub] kafka pull request: MINOR: Replace deprecated property in MirrorMa...

Github user sasakitoa closed the pull request at: https://github.com/apache/kafka/pull/733 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[jira] [Created] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

Jun Rao created KAFKA-3068: -- Summary: NetworkClient may connect to a different Kafka cluster than originally configured Key: KAFKA-3068 URL: https://issues.apache.org/jira/browse/KAFKA-3068 Project: Kafka

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084470#comment-15084470 ] Jun Rao commented on KAFKA-3068: [~enothereska], [~ijuma], [~guozhang], [~ewencp], [~hachi

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084481#comment-15084481 ] Jason Gustafson commented on KAFKA-3068: [~junrao] It does seem like an issue to m

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084478#comment-15084478 ] Guozhang Wang commented on KAFKA-3068: -- Even without this caching, we can still poten

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084507#comment-15084507 ] Jun Rao commented on KAFKA-3068: Yes, without this caching, in theory, this is still possi

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084516#comment-15084516 ] Guozhang Wang commented on KAFKA-3068: -- That is right... maybe we should just use boo

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Hi, Allen, A couple of minor comments on the KIP. 1. The version of the broker JSON string says 2. It should be 3. 2. The new version of UpdateMetadataRequest should be 2, instead of 1. Could you include the full wire protocol of version 2 of UpdateMetadataRequest and highlight the changed part?

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084609#comment-15084609 ] Ismael Juma commented on KAFKA-3068: Reverting to the bootstrap brokers in the config

RE: Reg. Kafka - Socket Communication from external system

Hi Gwen, Thanks for your reply. Our GPS device support TCP IP Socket communication with IP and Port. Kindly suggest KAKFA supports directly or do we need to write service layer for interface with KAFKA. Thanks & Regards, Basavaraj From: Gwen Shapira [mailto:g...@confluent.i

[GitHub] kafka pull request: MINOR: Update version to 0.9.0.1-SNAPSHOT

Github user ewencp closed the pull request at: https://github.com/apache/kafka/pull/727 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl