Re: kafka.common.LeaderNotAvailableException

2014-03-23 Thread Guozhang Wang
Hi Robert, This error is expected on the first few messages sending with a new topic, since the broker needs some time to create the topic and propagate the metadata. Please try keep sending and see if this exception goes away. If it is not transient, please let us know. Guozhang On Fri, Mar 21

Re: kafka.common.LeaderNotAvailableException

2014-03-23 Thread Guozhang Wang
BTW I think it is better if we can make such logging entries as WARN for the first few times, since they are not really ERRORs. Guozhang On Sun, Mar 23, 2014 at 10:33 AM, Guozhang Wang wrote: > Hi Robert, > > This error is expected on the first few messages sending with a new topic, > since th

Re: how to work arounf Kafka message length?

2014-03-23 Thread Guozhang Wang
Hi Vishal, You can tune the message.max.bytes configs on the broker side to allow 1mb messages along with compression: https://kafka.apache.org/documentation.html#configuration When you move on to 128mb then it would be a little hard to send the whole message at once, using Avro to serialize the