Hi Neha & Jun, I think we have found the reason of this bug. It is related to jdk versions. In the beginning, we use jdk1.6.20 to run the test.
Yesterday, we used jdk1.6.38. Everything is OK now. Thanks a lot for all your kind help and answer. On Thu, Mar 21, 2013 at 1:05 PM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > Do you mind filing a bug and attaching the reproducible test case there ? > > Thanks, > Neha > > On Wednesday, March 20, 2013, 王国栋 wrote: > > > Hi Jun, > > > > We use one thread with one sync produce to send data to broker > > (QPS:10k-15k, each log is about 1k bytes). The problem is reproduced. > > > > We have used Producer and SyncProducer in our test. The same Exception > > appears. > > > > Thanks. > > > > > > > > On Thu, Mar 21, 2013 at 12:19 PM, Jun Rao <jun...@gmail.com> wrote: > > > > > How many threads are you using? > > > > > > Thanks, > > > > > > Jun > > > > > > On Wed, Mar 20, 2013 at 7:33 PM, Yang Zhou <zhou.yang.h...@gmail.com> > > > wrote: > > > > > > > Sorry, I made a mistake, we use many threads producing at same time. > > > > > > > > > > > > 2013/3/20 Jun Rao <jun...@gmail.com> > > > > > > > > > How many producer instances do you have? Can you reproduce the > > problem > > > > with > > > > > a single producer? > > > > > > > > > > Thanks, > > > > > > > > > > Jun > > > > > > > > > > On Wed, Mar 20, 2013 at 12:29 AM, 王国栋 <wangg...@gmail.com> wrote: > > > > > > > > > > > Hi Jun, > > > > > > > > > > > > we do not use any compression in our test. > > > > > > > > > > > > We deploy producer and broker in the same machine. The problem > > still > > > > > > exists. We use sync producer, and send one message at a time(no > > batch > > > > > now). > > > > > > We find that when the qps reaches more than 40k, the exception > > > appears. > > > > > So > > > > > > I don't think it's the underlying system error. > > > > > > > > > > > > Any suggestions if we want to do some debug on kafka > > > > > > serialization/deserialization? > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Mar 20, 2013 at 12:10 AM, Jun Rao <jun...@gmail.com> > > wrote: > > > > > > > > > > > > > It basically means that the broker is expecting to read certain > > > > number > > > > > of > > > > > > > bytes in a buffer received from socket, but there are fewer > bytes > > > > than > > > > > > > expected in the buffer. Possible causes are (1) a bug in Kafka > > > > request > > > > > > > serialization/deserialization logic; (2) corruption in the > > > underlying > > > > > > > system such as network. > > > > > > > > > > > > > > BTW, did you enable compression in your producer? > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > On Mon, Mar 18, 2013 at 10:12 PM, Helin Xiang < > xkee...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > > > thanks Jun. > > > > > > > > > > > > > > > > we are using java producer. > > > > > > > > does the last exception > > > > > > > > "java.lang.IllegalArgumentException > > > > > > > > at java.nio.Buffer.limit(Buffer.java:266) > > > > > > > > " > > > > > > > > also means the broker received corrupted messages ? sorry i > am > > > not > > > > > > > > familiar with java nio. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Mar 19, 2013 at 12:58 PM, Jun Rao <jun...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > Hmm, both log4j messages suggest that the broker received > > some > > > > > > > corrupted > > > > > > > > > produce requests. Are you using the java producer? Also, we > > > have > > > > > seen > > > > > > > > that > > > > > > > > > network router problems caused corrupted requests before. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > On Mon, Mar 18, 2013 at 8:22 PM, Helin Xiang < > > -- > > Guodong Wang > > 王国栋 > > > -- Guodong Wang 王国栋
