Hi Team,
It is interesting to note that, the same code is working fine with kafka
0.8 release ( earlier to kafka-0.8.0-beta1-candidate1) with properties like
"broker.list" and "props.put("producer.type", "sync");" OR
"props.put("producer.type", "async");".
I suppose, the producer.type=sync is working wrongly in release
kafka-0.8.0-beta1-candidate1.
Team please have a look on this.
If my understanding is wrong please help to correct.
Thanks,
Yogesh Sangvikar
On Tue, Jun 25, 2013 at 4:38 PM, Jonathan Hodges <hodg...@gmail.com> wrote:
> We are able to telnet to each of the Kafka nodes from the producer so it
> doesn't appear to be a connectivity issue.
>
> DNVCOML-2D3FFT3:~ uhodgjo$ telnet x.x.x.168 9092
> Trying x.x.x.168...
> Connected to x.x.x.168.
> Escape character is '^]'.
> ^CConnection closed by foreign host.
> DNVCOML-2D3FFT3:~ uhodgjo$ telnet x.x.x.48 9092
> Trying x.x.x.48...
> Connected to x.x.x.48.
> Escape character is '^]'.
> ^CConnection closed by foreign host.
> DNVCOML-2D3FFT3:~ uhodgjo$ telnet x.x.x.234 9092
> Trying x.x.x.234...
> Connected to x.x.x.234.
> Escape character is '^]'.
> ^CConnection closed by foreign host.
> DNVCOML-2D3FFT3:~ uhodgjo$ telnet x.x.x.121 9092
> Trying x.x.x.121...
> Connected to x.x.x.121.
> Escape character is '^]'.
> ^CConnection closed by foreign host.
> DNVCOML-2D3FFT3:~ uhodgjo$ telnet x.x.x.236 9092
> Trying x.x.x.236...
> Connected to x.x.x.236.
> Escape character is '^]'.
> ^CConnection closed by foreign host.
> DNVCOML-2D3FFT3:~ uhodgjo$
>
>
> On Tue, Jun 25, 2013 at 4:57 AM, Jonathan Hodges <hodg...@gmail.com>
> wrote:
>
> > Hi Florin,
> >
> > I work with Yogesh so it is interesting you mention the
> > 'metadata.broker.list' property as this was the first error message we
> saw.
> > Consider the following producer code.
> >
> > Properties props = new Properties();
> > props.put("broker.list", "x.x.x.x:9092, x.x.x.x :9092, x.x.x.x :9092,
> > x.x.x.x :9092, x.x.x.x :9092");
> > props.put("producer.type", "sync");
> > props.put("compression.codec", "2"); //snappy
> > ProducerConfig config = new ProducerConfig(props);
> > producer = new Producer<byte[], byte[]>(config);
> >
> > This returns the following exception for the required, property
> > 'metadata.broker.list'.
> >
> > java.lang.IllegalArgumentException: requirement failed: Missing required
> > property 'metadata.broker.list'
> > at scala.Predef$.require(Predef.scala:145)
> > at
> >
> kafka.utils.VerifiableProperties.getString(VerifiableProperties.scala:158)
> > at kafka.producer.ProducerConfig.<init>(ProducerConfig.scala:66)
> > at kafka.producer.ProducerConfig.<init>(ProducerConfig.scala:56)
> > at com.pearson.firehose.KafkaProducer.<init>(KafkaProducer.java:21)
> > at com.pearson.firehose.KafkaProducer.main(KafkaProducer.java:40)
> >
> > So we just added 'metadata' prefix to the above 'broker.list' property
> and
> > this fixed this exception. However this is where we start to see this
> > producer retries error in the logs. Could there be some problem with the
> > value we are using for 'metadata.broker.list' which is preventing the
> > producer from connecting?
> >
> > Thanks,
> > Jonathan
> >
> >
> >
> > On Tue, Jun 25, 2013 at 1:12 AM, Florin Trofin <ftro...@adobe.com>
> wrote:
> >
> >> I got the same error but I think I had a different issue than you: My
> code
> >> was written for kafka 0.7 and when I switched to 0.8 I changed the
> >> "zk.connect" property to "metadata.broker.list" but left it with the
> same
> >> value (which was of course the zookeeper's host and port). In other
> words
> >> a "pilot error" :-) The snippet you provided doesn't seem to have this
> >> problem, but it is interesting that we got the same error (which would
> be
> >> nice if it can be customized depending on the actual problem: host
> >> unreachable, not responding, etc)
> >>
> >> F.
> >>
> >> On 6/24/13 10:55 PM, "Markus Roder" <roder.marku...@gmail.com> wrote:
> >>
> >> >We had this issue as well but never the less the message was enqueued
> >> >four times into the cluster. It would be great to get any hint on this
> >> >issue.
> >> >
> >> >regards
> >> >
> >> >--
> >> >Markus Roder
> >> >
> >> >Am 25.06.2013 um 07:18 schrieb Yogesh Sangvikar
> >> ><yogesh.sangvi...@gmail.com>:
> >> >
> >> >> Hi Jun,
> >> >>
> >> >> The stack trace we found is as follow,
> >> >>
> >> >> log4j:WARN No appenders could be found for logger
> >> >> (kafka.utils.VerifiableProperties).
> >> >> log4j:WARN Please initialize the log4j system properly.
> >> >> kafka.common.FailedToSendMessageException: Failed to send messages
> >> >>after 3
> >> >> tries.
> >> >> at
> >> >>
> >>
> >>
> >>kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala
> >> >>:90)
> >> >> at kafka.producer.Producer.send(Producer.scala:74)
> >> >> at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> >> >> at
> >> >>
> >>
> >>
> >>com.pearson.firehose.KafkaProducer.publishTinCanMessage(KafkaProducer.jav
> >> >>a:27)
> >> >> at
> >> com.pearson.firehose.KafkaProducer.main(KafkaProducer.java:44)
> >> >>
> >> >> Please let me know if you need the complete producer code.
> >> >>
> >> >> Thanks,
> >> >> Yogesh Sangvikar
> >> >>
> >> >>
> >> >> On Tue, Jun 25, 2013 at 10:05 AM, Jun Rao <jun...@gmail.com> wrote:
> >> >>
> >> >>> Could you attach the log before FailedToSendMessageException in the
> >> >>> producer? It should tell you the reason why the message can't be
> sent.
> >> >>>
> >> >>> Thanks,
> >> >>>
> >> >>> Jun
> >> >>>
> >> >>>
> >> >>> On Mon, Jun 24, 2013 at 9:20 PM, Yogesh Sangvikar <
> >> >>> yogesh.sangvi...@gmail.com> wrote:
> >> >>>
> >> >>>> Hi Team,
> >> >>>>
> >> >>>> We are using kafka-0.8.0-beta1-candidate1 release. (
> >> >>>> https://github.com/apache/kafka/tree/0.8.0-beta1-candidate1).
> >> >>>> While running producer with following configuration, we found an
> >> issue
> >> >>>> "kafka.common.
> >> >>>> FailedToSendMessageException: Failed to send messages after 3
> tries",
> >> >>>>
> >> >>>> We are using default broker configurations.
> >> >>>>
> >> >>>> Code snippet:
> >> >>>>
> >> >>>> private Producer<byte[], byte[]> producer = null;
> >> >>>>
> >> >>>> public KafkaProducer() {
> >> >>>> Properties props = new Properties();
> >> >>>> props.put("metadata.broker.list", "broker1:9092<
> >> >>>> http://10.252.8.168:9092>
> >> >>>> ,broker2:9092 <http://10.252.8.48:9092>,broker3:9092<
> >> >>>> http://10.252.8.234:9092>
> >> >>>> ,broker4:9092 <http://10.252.8.121:9092>,broker5:9092<
> >> >>>> http://10.252.8.236:9092>
> >> >>>> ");
> >> >>>> *props.put("producer.type", "sync");*
> >> >>>> ProducerConfig config = new ProducerConfig(props);
> >> >>>> producer = new Producer<byte[], byte[]>(config);
> >> >>>> }
> >> >>>>
> >> >>>> public void publishTinCanMessage(String message, int event) throws
> >> >>>> Exception {
> >> >>>> KeyedMessage<byte[], byte[]> data = new KeyedMessage<byte[],
> >> >>>> byte[]>("tin_can_topic",
> >> >>>> (String.valueOf(event%3)).getBytes(),message.getBytes());
> >> >>>> producer.send(data);
> >> >>>>
> >> >>>> }
> >> >>>> ......
> >> >>>>
> >> >>>> Found issue:
> >> >>>> kafka.common.
> >> >>>> *FailedToSendMessageException: Failed to send messages after 3
> >> tries.*
> >> >>>> at
> >> >>>
> >>
> >>
> >>>kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scal
> >> >>>a:90)
> >> >>>> at kafka.producer.Producer.send(Producer.scala:74)
> >> >>>> at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> >> >>>> at
> >> >>>
> >>
> >>
> >>>com.pearson.firehose.KafkaProducer.publishTinCanMessage(KafkaProducer.ja
> >> >>>va:27)
> >> >>>> at
> >> >>>>com.pearson.firehose.KafkaProducer.main(KafkaProducer.java:44)
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> But. with *props.put("producer.type", "async"); *the producer was
> >> >>>>working
> >> >>>> fine and generating messages.
> >> >>>>
> >> >>>> Could you please help us to understand is there any configuration
> >> >>>>missing
> >> >>>> or is there any issue with "producer.type=sync"?
> >> >>>>
> >> >>>>
> >> >>>> Thanks in advance.
> >> >>>>
> >> >>>> Thanks,
> >> >>>> Yogesh Sangikar
> >> >>>
> >>
> >>
> >
>
