Re: getting Error name

That’s what I want. Thanks, Jun On Feb 24, 2014, at 2:09 PM, Jun Rao wrote: > You can use ErrorMapping.exceptionFor(short), which returns a Throwable. > > Thanks, > > Jun > > > On Sun, Feb 23, 2014 at 5:43 PM, Bongyeon Kim wrote: > >> Hi, everyone. >> >> I'm using Kafka 0.8 with Java AP

Re: producer not pushing the data into broker

Hi, thanks for the input. I will try to increase the number of retries and check. Thanks Arjun Narasimha kota On Monday 24 February 2014 10:44 AM, Jun Rao wrote: If those errors only show up transiently when brokers are started, then it's normal. It takes a bit of time for metadata to be prob

Re: Reg Exception in Kafka

Yeah, I've only skimmed this, but I think I might have something. All non-vpc type ec2 nodes come with an external IP address and an internal IP address. The external IP address is what grants the node access to the internet--makes it publicly routable. The mechanism by which the external IP addre

Re: Reg Exception in Kafka

Hmm, then I am not sure what happened. Anyone with EC2 experience can help? Thanks, Jun On Sun, Feb 23, 2014 at 6:00 PM, Balasubramanian Jayaraman (Contingent) < balasubramanian.jayara...@autodesk.com> wrote: > The ports are already added in the security group. I am able to telnet > from the s

Re: producer not pushing the data into broker

If those errors only show up transiently when brokers are started, then it's normal. It takes a bit of time for metadata to be probably to the newly started brokers. You may have to increase # retries to avoid the failure in the producer. Thanks, Jun On Sun, Feb 23, 2014 at 6:33 PM, Arjun wrot

Re: getting Error name

You can use ErrorMapping.exceptionFor(short), which returns a Throwable. Thanks, Jun On Sun, Feb 23, 2014 at 5:43 PM, Bongyeon Kim wrote: > Hi, everyone. > > I'm using Kafka 0.8 with Java API. > > > Is there any Error Mapping class or constants? I found Kafka Protocol page > in wiki. There is

Re: producer not pushing the data into broker

Hi, No i have 3 kafka brokers running, but in the same system. I tried with the replication factor of 1 but it gives the same result. Thanks Arjun Narasimha Kota On Monday 24 February 2014 04:21 AM, Jun Rao wrote: With only one broker, do you really want to have replication factor 2? Maybe yo

RE: Reg Exception in Kafka

The ports are already added in the security group. I am able to telnet from the same machine I am running the producer/consume test. Is there any configuration I missed? Thanks Bala -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Saturday, February 22, 2014 12:10 AM To:

getting Error name

Hi, everyone. I'm using Kafka 0.8 with Java API. Is there any Error Mapping class or constants? I found Kafka Protocol page in wiki. There is error code table, but I want to get error name by error code in Java Api. For example, I can getting error code by *fetchResponse.errorCode(TOPIC, PARTIT

Re: New Consumer API discussion

Robert, For the push orient api, you can potentially implement your own MessageHandler with those methods. In the main loop of our new consumer api, you can just call those methods based on the events you get. Also, we already have an api to get the first and the last offset of a partition (getOf

Re: Host resolution issue solution

Thanks for the info. I update FAQ ( https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whycan'tmyconsumers/producersconnecttothebrokers?) to reflect this. Jun On Sun, Feb 23, 2014 at 4:52 AM, Krishna Raj wrote: > Hi All, > > I have banged my head for nearly 3 hours trying to find out why

Re: Unexpected broker election

Have you looked at https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whypartitionleadersmigratethemselvessometimes ? Thanks, Jun On Sat, Feb 22, 2014 at 2:06 PM, Andrew Otto wrote: > Yeah, I can do that, but I'd prefer if the first broker didn't drop out of > the ISR in the first plac

Re: producer not pushing the data into broker

With only one broker, do you really want to have replication factor 2? Maybe you could retry with replication factor 1. Thanks, Jun On Sat, Feb 22, 2014 at 12:54 AM, Arjun wrote: > Hi please find below the o/p of the list and the console producer. There > are no errors in the state change l

Powered By Page

Hey folks, quick community shout out to anyone using Apache Kafka in production to update the powered by page https://cwiki.apache.org/confluence/display/KAFKA/Powered+By or send me what you want on the page and I can do the updates to it. If you wan to update the page yourself but don't have acce

Re: New Consumer API discussion

We use kafka as a durable buffer for 3rd party event traffic. It acts as the event source in a lambda architecture. We want it to be exactly once and we are close, though we can lose messages aggregating for Hadoop. To really tie this all together, I think there should be an Apache project to

Re: node.js native bindings

It appears compression (gzip, snappy) is not supported in the librdkafka version (8.0.0) used in this binding. Is there any intention to update this to use a more recent librdkafka version? If so it would be great as there does not seem to be any other node-kafka solution that supports Kafka 0.8 wi

Re: Delayed Queue

Thanks Martin for the explanation. Regards, Jagan On Sun, 23 Feb 2014 18:19:48 +0530 Martin Kleppmann wrote Hi Jagan, unfortunately Kafka doesn't have the same TTL feature as you find in RabbitMQ. This is because Kafka and RabbitMQ have a fundamentally di

Host resolution issue solution

Hi All, I have banged my head for nearly 3 hours trying to find out why my producer code did not enqueue the message. And later found a very simple issue. I dont want anyone else to waste their time in this and hence this email. 1) Broker is in a physical Linux box. The broker was registered usi

Re: Delayed Queue

Hi Jagan, unfortunately Kafka doesn't have the same TTL feature as you find in RabbitMQ. This is because Kafka and RabbitMQ have a fundamentally different design: - RabbitMQ brokers individually track the status of each message (whether it has been acked by a consumer, when its TTL expires, etc

Re: Delayed Queue

Thanks Aniket for your suggestion. I am new to Kafka, so wonder if there is something like a TTL and deadletter exchange similar to RabbitMQ, which kind of does the job for me. That kind of simplifies things. Regards, Jagan On Sun, 23 Feb 2014 15:44:46 +0530 Aniket Bhatnagar

Re: Delayed Queue

Alright. Here is an approach that I can think of. Create a topic for each delay and produce message to the appropriate topic. Each produced message should have a timestamp at which it was pushed to the queue. At the consumer side, fetch a message from a partition and compare the message timestamp w

Re: Delayed Queue

Hi Aniket, I am looking at "Finite set of delays" and messages will use one on the defined set. Regards, Jagan On Sun, 23 Feb 2014 15:13:02 +0530 Aniket Bhatnagar wrote Hi Jagan Are you expecting each message to have the same delay? Or is there a f

Re: Delayed Queue

Also I am concerned about the Producer level batching and the data loss that this may lead to. Can I turn off the producer level batching completely. If yes, what is the performance that I am paying for this? Regards, Jagan On Sun, 23 Feb 2014 14:13:12 +0530 Jagan Ranganathan

Re: Delayed Queue

Hi Jagan Are you expecting each message to have the same delay? Or is there a finite set of delays? Or messages can arbitrarily any delay? Thanks, Aniket On 23 Feb 2014 14:13, "Jagan Ranganathan" wrote: > Dear All, > > I am looking for a Delayed Queue implementation for my distributed system >

Delayed Queue

Dear All, I am looking for a Delayed Queue implementation for my distributed system requirement. Would like to know if I can achieve the same using Kafka. Regards, Jagan