Want to speak at Apache Kafka meetup at LinkedIn on February 23?

Hi Friends of Kafka, Doing anything cool with Kafka? Looking to share?! LinkedIn will host an Apache Kafka meetup in Mountain View CA on the evening of Tuesday, February 23. We currently have 2-talks figured out but would like to

Re: Mirrormaker stops consuming

Thanks Raja. We will try to continue to investigate and any more updates we will post to this thread. Thanks, Debbie From: Rajasekar Elango mailto:rela...@salesforce.com>> Date: Monday, February 8, 2016 at 10:32 AM To: "Yu, Debbie" mailto:deborah...@intuit.com>>, "users@kafka.apache.org

Re: NotLeaderForPartitionException: This server is not the leader for that topic-partition.

Sorry for reviving this old mailing list discussion. I'm facing a similar issue while running a load test with many small topics (100 topics) with 4 partitions each. There is also a Flink user who's facing the issue: https://issues.apache.org/jira/browse/FLINK-3066 Are you also writing into many

Mirror maker stops working (0.8.2.1)

Hello, We have been seeing consistent issue mirroring between our DataCenters. Below is our setup Source: AWS (13 Brokers) Destination: OTHER-DC (20 Brokers) Topic: mirrortest (source: 260 partitions, destination: 200 partitions) Connectivity: AWS Direct Connect (max 6Gb

Re: FW: Mirrormaker stops consuming

Debbie, In our case, the issue was with custom SSL implementation and we need to fix a bug related to buffer over flow handling. This may not be applicable to you, if you are using open source kafka. The best way to troubleshoot this problem would be to take couple of thread dumps with few second

Re: Session timeout and heartbeat interval

Hey Jens, The heartbeat response is used by the coordinator to tell group members that the group needs to rebalance. For example, if a new member joins the consumer group, then the coordinator will wait for the heartbeat from each member and set a REBALANCE_NEEDED error code in the response. Hence

RE: WELCOME

gradle going fubar as it recurses thru core folder as soon as I replaced with maven 3.3 pom.xml's everything started to work Thanks/ Martin Gainty __ _ _ _ _ _ ___ _ _

Re: WELCOME

If it's taking that long, you may be working on hardware (or a VM?) which is too underpowered to run some of the tests reliably. The README.md has instructions for how to build different components. In your case you want ./gradlew core:test Depending on your hardware, you may want to adjust the

Re: Possible bug in Kafka Connect - Schema modified internally

If you're using the JsonConverter, it looks like you're seeing https://issues.apache.org/jira/browse/KAFKA-3055. It's been fixed and will be included in 0.9.0.1 (and 0.9.1.0, which is trunk). -Ewen On Sun, Feb 7, 2016 at 8:34 PM, Shiti Saxena wrote: > Hi, > > When defining a Kafka pipeline I re

Re: Caused by: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.

It seems that you put in the wrong port in the following statement. Kafka's default port is 9092. 2181 is for Zookeeper. props.put("bootstrap.servers", "localhost:2181"); Thanks, Jun On Mon, Feb 8, 2016 at 4:06 AM, Bhargav Maddikera < bhargav.maddik...@altimetrik.com> wrote: > Hi, > > I try

Re: Regarding issue in Kafka-0.8.2.2.3

I export my JMX_PORT setting in the kafka-server-start.sh script and have not run into any issues yet. On Mon, Feb 8, 2016 at 9:01 AM, Manikumar Reddy wrote: > kafka scripts uses "kafka-run-class.sh" script to set environment variables > and run classes. So if you set any environment variable >

Re: Regarding issue in Kafka-0.8.2.2.3

kafka scripts uses "kafka-run-class.sh" script to set environment variables and run classes. So if you set any environment variable in"kafka-run-class.sh" script, then it will be applicable to all the scripts. So try to set different JMX_PORT in kafka-topics.sh. On Mon, Feb 8, 2016 at 9:24 PM, Shi

RE: gradle

a not so kind person dumped all the jars into kafka-0.9.0.0-src/core/build/dependant-libs-2.10.5 folder as soon as installed all of the jars to maven repository kafka core compiled and produced a jar Pom attached Martin __ _ _ _

Log message key content into LOG4j

Hi Is it possibile to log message header / key into server logs ( Log4j ) on message broker ? I would like to track message after a few years , so I need to prove that message exists on message broker in some particular time. my case : Publisher log information into business log (e.g. log4j) th

Caused by: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.

Hi, I try executing the code Properties props = new Properties(); props.put("bootstrap.servers", "localhost:2181"); props.put("metadata.broker.list","localhost:9092"); props.put("request.required.acks", "1"); props.put("retries", 4)

Re: Regarding issue in Kafka-0.8.2.2.3

Hi, In order to get metrics through JMX, we have exported JMX_PORT=8099. But when we are trying to delete the topics from kafka, we are getting following issue: Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 8099; nested exception is: java.net

Regarding issue in Kafka-0.8.2.2.3

Hi, In order to get metrics through JMX, we have exported JMX_PORT=8099. But when we are trying to delete the topics from kafka, we are getting following issue: Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 1234; nested exception is: java.net

RE: gradle

gently placed gradle into the round file and used normal maven to compile kafka core when I encountered this error: import joptsimple.*; kafka/kafka-0.9.0.0-src/core/src/main/scala/kafka/tools/KafkaMigrationTool.java:[20,1] package joptsimple does not exist does anyone know where joptsimple is s

Re: Session timeout and heartbeat interval

Hi again, A somewhat related question is also how the heartbeat interval and session timeout relates to the poll timeout. Must the poll timeout always be lower than the heartbeat interval? Cheers, Jens On Monday, February 8, 2016, Jens Rantil wrote: > Hi, > > I am trying to wrap my head around

RE: WELCOME

gradle build ran all weekend ..specificallykafka>gradlew build > Building 29% > :core:test > 365 tests completed, 58 failed hung at the line above I think for now I just want to run "test phase" against "core folder"...How do I do that? Thanks! Martin _

Session timeout and heartbeat interval

Hi, I am trying to wrap my head around the difference between " session.timeout.ms" and "heartbeat.interval.ms". Clearly they are somewhat tied to the same logic since heartbeat value should be less than 1/3 of the session timeout. Let me add some specific questions: - Will a session be consi

Re: Kafka 0.9.0.1 plan

Hi Pierre, My take is that it is unlikely that KAFKA-3006 can be part of 0.9.0.1 as KIP-45 is still being discussed. Also, the compatibility implications mean that 0.9.1.0 is a more likely target. Ismael On Fri, Feb 5, 2016 at 11:41 PM, Pierre-Yves Ritschard wrote: > Hi, > > Since it's still e