Hi Victor, two points: - Based on the backtrace, you are using a very old version of Sarama. You might have better luck using a more recent stable version. - Are you setting `MaxBufferTime` in the configuration to 0 or a very small value? If so the loop will spin on that timer. Try making this value larger.
Evan On Wed, Apr 15, 2015 at 12:35 PM, Victor L <vlyamt...@gmail.com> wrote: > I am using sarama "golang" kafka 1.8.1 client ( > https://github.com/Shopify/sarama) to send messages to message queue once > in 3secs and this task drives cpu consumption to 130% on the quad-cpu > blade; The number stays this high regardless of number of > partitions/consumers.... According to results of profiling with 'pprof', > most of the cycles are spent in producer.go/NewBrokerProducer; > Total: 47297 samples > 5947 12.6% 12.6% 15013 31.7% selectgo > 3617 7.6% 20.2% 3617 7.6% runtime.xchg > 2819 6.0% 26.2% 28506 60.3% github.com/Shopify/sarama.func·008 > <http://github.com/Shopify/sarama.func%C2%B7008> > <http://github.com/Shopify/sarama.func%C2%B7008> > ---------------------------------------------------------------------- > NewBrokerProducer > ----------------------------------------------------------- > 229: go func() { > . . 230: timer := time.NewTimer(p.config.MaxBufferTime) > . . 231: var shutdownRequired bool > . . 232: wg.Done() > . . 233: for { > 208 17598 234: select { > 153 749 235: case <-bp.flushNow: > . . 236: if shutdownRequired = bp.flush(p); > shutdownRequired { > . . 237: goto shutdown > . . 238: } > 2230 3144 239: case <-timer.C: > 144 3606 240: if shutdownRequired = > bp.flushIfAnyMessages(p); shutdownRequired { > . . 241: goto shutdown > > > wonder if there's some known issue with this method or if anyone already > seen it before... > Thank you, >
