I am not sure how your project was setup. But I think it depends on what log4j property file you specified when you started your application. Can you check if you have log4j appender defined and the loggers are directed to the correct appender?
Thanks, Jiangjie (Becket) Qin On 7/15/15, 8:10 AM, "Vadim Bobrov" <vadimbob...@gmail.com> wrote: >Thanks Jiangjie, > >unfortunately turning trace level on does not seem to work (any log level >actually) I am using log4j2 (through slf4j) and despite including log4j1 >bridge and these lines: > ><Logger name="org.apache.kafka" level="trace"/> ><Logger name="kafka" level="trace"/> > >in my conf file I could not squeeze out any logging from kafka. Logging >for >all other libs (like zookeeper e.g.) work perfectly. Am I doing something >wrong? > > >On Tue, Jul 14, 2015 at 6:55 PM, Jiangjie Qin <j...@linkedin.com.invalid> >wrote: > >> Hi Vadim, >> >> Can you turn on trace level logging on your consumer and search for >> "offset commit response² in the log? >> Also maybe take a look at the log to see if there is any exception >>thrown. >> >> Thanks, >> >> Jiangjie (Becket) Qin >> >> On 7/14/15, 11:06 AM, "Vadim Bobrov" <vadimbob...@gmail.com> wrote: >> >> >just caught this error again. I issue commitOffsets - no error but no >> >committng offsets either. __consumer_offsets watching shows no new >> >messages >> >either. Then in a few minutes I issue commitOffsets again - all >>committed. >> >Unless I am doing something terribly wrong this is very unreliable >> > >> >On Tue, Jul 14, 2015 at 1:49 PM, Joel Koshy <jjkosh...@gmail.com> >>wrote: >> > >> >> Actually, how are you committing offsets? Are you using the old >> >> (zookeeperconsumerconnector) or new KafkaConsumer? >> >> >> >> It is true that the current APIs don't return any result, but it >>would >> >> help to check if anything is getting into the offsets topic - unless >> >> you are seeing errors in the logs, the offset commit should succeed >> >> (if you are indeed explicitly committing offsets). >> >> >> >> Thanks, >> >> >> >> Joel >> >> >> >> On Tue, Jul 14, 2015 at 12:19:01PM -0400, Vadim Bobrov wrote: >> >> > Thanks, Joel, I will but regardless of my findings the basic >>problem >> >>will >> >> > still be there: there is no guarantee that the offsets will be >> >>committed >> >> > after commitOffsets. Because commitOffsets does not return its exit >> >> status, >> >> > nor does it block as I understand until offsets are committed. In >> >>other >> >> > words, there is no way to know that it has, in fact, commited the >> >>offsets >> >> > >> >> > or am I missing something? And then another question - why does it >> >>seem >> >> to >> >> > depend on the number of consumed messages? >> >> > >> >> > On Tue, Jul 14, 2015 at 11:36 AM, Joel Koshy <jjkosh...@gmail.com> >> >> wrote: >> >> > >> >> > > Can you take a look at the kafka commit rate mbean on your >>consumer? >> >> > > Also, can you consume the offsets topic while you are committing >> >> > > offsets and see if/what offsets are getting committed? >> >> > > (http://www.slideshare.net/jjkoshy/offset-management-in-kafka/32) >> >> > > >> >> > > Thanks, >> >> > > >> >> > > Joel >> >> > > >> >> > > On Tue, Jul 14, 2015 at 11:12:03AM -0400, Vadim Bobrov wrote: >> >> > > > I am trying to replace ActiveMQ with Kafka in our environment >> >> however I >> >> > > > have encountered a strange problem that basically prevents from >> >>using >> >> > > Kafka >> >> > > > in production. The problem is that sometimes the offsets are >>not >> >> > > committed. >> >> > > > >> >> > > > I am using Kafka 0.8.2.1, offset storage = kafka, high level >> >> consumer, >> >> > > > auto-commit = off. Every N messages I issue commitOffsets(). >>Now >> >> here is >> >> > > > the problem - if N is below a certain number (180 000 for me) >>it >> >> works >> >> > > and >> >> > > > the offset is moving. If N is 180 000 or more the offset is not >> >> updated >> >> > > > after commitOffsets >> >> > > > >> >> > > > I am looking at offsets using kafka-run-class.sh >> >> > > > kafka.tools.ConsumerOffsetChecker >> >> > > > Any help? >> >> > > >> >> > > >> >> >> >> >> >>
