Re: Too much log for kafka.common.KafkaException

2014-10-18 Thread xingcan
itions read in the > FetchRequest (i.e. make sure FetchRequests address fewer > TopicAndPartitions, maybe putting each TopicAndPartition in its own > request). An alternative would be to use more recent offsets (i.e. don't > start from the oldest data available in Kafka). A recent

Too much log for kafka.common.KafkaException

2014-10-18 Thread xingcan
at kafka.network.Processor.write(SocketServer.scala:375) at kafka.network.Processor.run(SocketServer.scala:247) at java.lang.Thread.run(Thread.java:745) -- *Xingcan*

Re: About speed contorl in Kafka

2013-12-13 Thread xingcan
ghput is far higher > than the producer->broker throughput. > > > 2013/12/12 xingcan > > > Guozhang, > > > > Thanks for your prompt replay. I got two 300GB SAS disks for each borker. > > At peak time, the produce speed for each broker is about 70MB/s. &g

Re: About speed contorl in Kafka

2013-12-11 Thread xingcan
ozhang > > > On Wed, Dec 11, 2013 at 9:09 PM, xingcan wrote: > > > Hi, > > > > In my application, the produce speed could be very high at some specific > > time in a day while return > > to a low speed at the rest of time. Frequently, my data logs are flush

About speed contorl in Kafka

2013-12-11 Thread xingcan
speed seems to be difficult and adding space for log files are also not a ultimate solution. Any suggestions for this problem? Are there any speed control mechanism in Kafka? Thanks. -- *Xingcan*

Re: How to deal with error code in FetchResponse

2013-10-23 Thread xingcan
nks, > Neha > > > On Wed, Oct 23, 2013 at 1:36 AM, xingcan wrote: > >> Neha, >> >> Thanks for your prompt reply. And I got another two questions. As I wrote >> before, >> my mechanism is to add all partitions belonging to the same leader broker >> t

Re: How to deal with error code in FetchResponse

2013-10-23 Thread xingcan
> Thanks, > Neha > On Oct 21, 2013 11:27 PM, "xingcan" wrote: > > > Hi, > > > > After migrating from 0.72 to 0.8, I still use SimpleConsumer to construct > > my own consumer. By FetchRequestBuilder, I add all partitions belonging > to > > the s

How to deal with error code in FetchResponse

2013-10-21 Thread xingcan
little hard to retrieve for I must iterate all these partitions to check. Is there any tips or suggestions to deal with this or maybe the API provided by FetchResponse could be little changed? -- *Xingcan*

Re: A "Java heap space" question

2012-12-27 Thread xingcan
Jun, Out messages are not plain text. Most of them are JPEG files. I'm not sure if compression will be useful. 2012/12/28 Jun Rao > Are messages compressed? If not, compressing them could save java heap > space. > > Thanks, > > Jun > > > On Thu, Dec 27,

Re: A "Java heap space" question

2012-12-27 Thread xingcan
ur JVM heap? You may need a bigger heap size. > > Thanks, > > Jun > > On Wed, Dec 26, 2012 at 5:41 PM, xingcan wrote: > > > Hi, > > > > I run 'kafka-0.7.2' on a single node and today I got some error logs > below. > > > > ... > >