Re: kafka producer failed

2015-07-26 Thread Yi Pan
Great to hear that! Cheers! -Yi On Sun, Jul 26, 2015 at 10:53 PM, Job-Selina Wu wrote: > Hi, Yi: > > you are right. After I add the producer.close(); the bug is fixed now. > > Thanks a lot! > > Selina > > On Sun, Jul 26, 2015 at 10:28 PM, Yi Pan wrote: > > > Hi, Selina, > > > > Did you for

Re: kafka producer failed

2015-07-26 Thread Job-Selina Wu
Hi, Yi: you are right. After I add the producer.close(); the bug is fixed now. Thanks a lot! Selina On Sun, Jul 26, 2015 at 10:28 PM, Yi Pan wrote: > Hi, Selina, > > Did you forget to close your Kafka producer in your Http servlet? If you > create a new Kafka producer in each Http request

Re: kafka producer failed

2015-07-26 Thread Yi Pan
Hi, Selina, Did you forget to close your Kafka producer in your Http servlet? If you create a new Kafka producer in each Http request and do not close the producer, that might cause this problem. -Yi On Sun, Jul 26, 2015 at 9:25 PM, Job-Selina Wu wrote: > Hi, Yan: > > the kaka.log got *jav

Re: kafka producer failed

2015-07-26 Thread Job-Selina Wu
Hi, Yan: the kaka.log got *java.io.IOException: Too many open files in system* Some detail list below. Sincerely, Selina [2015-07-26 21:07:01,500] INFO Verifying properties (kafka.utils.VerifiableProperties) [2015-07-26 21:07:01,545] INFO Property broker.id is overridden to 0 (kafka.utils.V

Re: kafka producer failed

2015-07-26 Thread Yan Fang
You may check the Kafka.log to see what's inside Yan Fang > On Jul 26, 2015, at 2:01 AM, Job-Selina Wu wrote: > > The exception is below: > > kafka.common.FailedToSendMessageException: Failed to send messages after 3 > tries. > kafka.common.FailedToSendMessageException: Failed to send message

Re: kafka producer failed

2015-07-26 Thread Job-Selina Wu
The exception is below: kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90) at kafka.producer.Producer.se

Re: kafka producer failed

2015-07-26 Thread Job-Selina Wu
Hi, Yan: My Http Server send message to Kafka. The server.log at deploy/kafka/logs/server.log shown : [2015-07-26 00:33:51,910] INFO Closing socket connection to /127.0.0.1. (kafka.network.Processor) [2015-07-26 00:33:51,984] INFO Closing socket connection to /127.0.0.1. (kafka.network.Pro

Re: kafka producer failed

2015-07-26 Thread Yan Fang
You are giving the Kafka code and the Samza log, which does not make sense actually... Fang, Yan yanfang...@gmail.com On Sat, Jul 25, 2015 at 10:31 PM, Job-Selina Wu wrote: > Hi, Yi, Navina and Benjamin: > > Thanks a lot to spending your time to help me this issue. > > The configuration

Re: kafka producer failed

2015-07-25 Thread Job-Selina Wu
Hi, Yi, Navina and Benjamin: Thanks a lot to spending your time to help me this issue. The configuration is below. Do you think it could be the configuration problem? I tried props.put("request.required.acks", "0"); and props.put("request.required.acks", "1"); both did not work.

Re: kafka producer failed

2015-07-24 Thread Benjamin Black
what are the log messages from the kafka brokers? these look like client messages indicating a broker problem. On Fri, Jul 24, 2015 at 1:18 PM, Job-Selina Wu wrote: > Hi, Yi: > > I am wondering if the problem can be fixed by the parameter " > max.message.size" at kafka.producer.ProducerCo

Re: kafka producer failed

2015-07-24 Thread Yi Pan
Hi, Selina, I assume that you were referring to "max.message.bytes" in Kafka producer config? There is no "max.message.size" config. If you were referring to the "max.message.bytes", it has nothing to do w/ the number of messages in a topic, it is the limit on a single message size in bytes in Kaf

Re: kafka producer failed

2015-07-24 Thread Job-Selina Wu
Hi, All: Do you think it could be caused by memory, virtual memory size? Sincerely, Selina On Fri, Jul 24, 2015 at 1:54 PM, Job-Selina Wu wrote: > Hi, Navina: > > Thanks for your reply: the files are listed below: > > Your help is highly appreciated. > > Sincerely, > Selina > >

Re: kafka producer failed

2015-07-24 Thread Job-Selina Wu
Hi, Navina: Thanks for your reply: the files are listed below: Your help is highly appreciated. Sincerely, Selina The producer.properties for Kafka: # Producer Basics # # list

Re: kafka producer failed

2015-07-24 Thread Navina Ramesh
Hi Selina, Looks like the attachment got filtered. Can you past the config in the email or at pastebin? Thanks! Navina On Fri, Jul 24, 2015 at 1:18 PM, Job-Selina Wu wrote: > Hi, Yi: > > I am wondering if the problem can be fixed by the parameter " > max.message.size" at kafka.producer.P

Re: kafka producer failed

2015-07-24 Thread Job-Selina Wu
Hi, Yi: I am wondering if the problem can be fixed by the parameter " max.message.size" at kafka.producer.ProducerConfig for the topic size? My Http Server send message to Kafka. The last message shown on console is "message=timestamp=06-20-2015 id=678 ip=22.231.113.68 browser=Safari

Re: kafka producer failed

2015-07-23 Thread Yi Pan
Hi, Selina, Your question is not clear. {quote} When the messages was send to Kafka by KafkaProducer, It always failed when the message more than 3000 - 4000 messages. {quote} What's failing? The error stack shows errors on the consumer side and you were referring to failures to produce to Kafka.

Re: kafka producer failed

2015-07-23 Thread Job-Selina Wu
Hi, After I got the error in previous email. I try to check the content of the topic. It show the error below. However when I stop Samza and re-run it again, it will be fine. Does anyone know What was the problem? [2015-07-23 17:50:08,391] WARN [console-consumer-83311_Selinas-MacBook-Pro.loc