Hi Jun, thanks for you answer. Unfortunately the size won't help much, I'd like to see the actual message data.
By the way what are the things to consider when deciding on `message.max.bytes` value? On Wed, Aug 27, 2014 at 9:06 PM, Jun Rao <jun...@gmail.com> wrote: > The message size check is currently only done on the broker. If you enable > trace level logging in RequestChannel, you will see the produce request, > which includes the size of each partition. > > Thanks, > > Jun > > > On Wed, Aug 27, 2014 at 4:40 PM, Alexis Midon < > alexis.mi...@airbedandbreakfast.com> wrote: > > > Hello, > > > > my brokers are reporting that some received messages exceed the > > `message.max.bytes` value. > > I'd like to know what producers are at fault but It is pretty much > > impossible: > > - the brokers don't log the content of the rejected messages > > - the log messages do not contain the IP of the producers > > - on the consumer side, no exception is thrown (afaik it is because Ack-0 > > is used). The only kind of notification is to closed the connection. > > > > [1] Do you have any suggestions to track down the guilty producers or > find > > out the message content? > > > > Even though it makes total sense to have the limit defined and applied on > > the brokers, I was thinking that this check could also be applied by the > > producers. Some google results suggest that `message.max.bytes` might be > > used by the producers but I can't find any trace of that behavior in the > > code. > > > > The closest thing I have is > > > > > https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/producer/SyncProducer.scala#L67 > > but it simply logs the message size and content and the log level is > trace. > > > > [2] could you please confirm if such a producer-side check exists? > > > > > > thanks! > > > > Alexis > > >
