Hi, I'd like to take a closer look at this, but I need more details.
Are you able to come up with a reproducer for this issue? Additionally,
details about your Kafka setup would be appreciated: # of partitions, any
broker setting you might have, etc.
Kind regards
On Sun, Jun 26, 2022 at 3:24 PM Rohan Emmanuel <rohan.emmanue...@gmail.com>
wrote:
> hello Claus,
> Thank you for responding. i tried with camel 3.17.0 and kafka clients 3.2.0
> , but I still see the CPU busy, The process is busy at 99% even i am not
> sending my messages to the topics.
>
> Looks like Kafka clients calls(polling) are doing CPU-intensive operations
> repeatedly. Any configuration which can reduce the number of Kafka Clients
> calls?
>
>
>
> Attached below is the Real Time process Monitor output, $sr10 & $sr11
> running on CPUs 2 & 3 respectively, are my processes running Kafka
> endpoint.
>
>
>
> Cpus Cp hh:mm ss Busy Sec QLen Disp Disk Chit Swap MLock Pcb PcbX
> -------- -- -------- ---% --- ---- ------ ----- ------ ---- ----% --- ----
> 0 18:57:29 10 998 1 12.88
> 29 359
> 1 18:57:29 10 843 12.43
> 21 237
> * 2 *18:57:29 *18 * 10 1 1191 326
> 11.27 12 276
> * 3 *18:57:29 * 17 *10 1 1168
> 11.11 10 465
> 4 18:57:29 10 821 6.15 12
> 149
> 5 18:57:29 10 832 6.24
> 9 153
> 6 18:57:29 10 805 7.79 9
> 141
> Process Cpu,Pin Pri Busy% Name RPM T0877(16SEP16) ET=10.1 Top User
> -------- ------- --- ----- -------- -------------------------- --- -------
> 3,612 170 *99.62* *$SR11 * /jdk110_l11/bin/java 1
> 201,4
> 2,1073 170 * 98.59 $SR10* /jdk110_l11/bin/java 2
> 201,4
> 2,1076 170 .99 /jdk110_l11/bin/javahelper 3 201,4
> 2,1077 170 .99 /jdk110_l11/bin/javahelper 4 201,4
> 2,1074 170 .98 /jdk110_l11/bin/javahelper 5 201,4
>
>
>
> The route looks like :
>
> <route id="kafka_endpoint-kafka">
> <from
>
> uri="kafka:kavi-reqs?brokers=<IP>:9092&fetchMinBytes=20480&reconnectBackoffMaxMs=10000&reconnectBackoffMs=2000&groupId=APIG"/>
> <doTry>
> <process ref="api.gway.kafka_endpoint.kafka.request"/>
> <to uri="custom_component"/>
>
> <doCatch >
> <exception>java.lang.Exception</exception>
>
>
> <!--setBody-->
> <transform>
> <simple>Error reported: ${exception.message} - cannot process this message.
> </simple>
>
> </transform>
> <!--/setBody-->
>
> <log message="${exception.stacktrace}" />
>
>
> </doCatch>
> </doTry>
>
>
>
> On Sat, Jun 25, 2022 at 6:40 PM Claus Ibsen <claus.ib...@gmail.com> wrote:
>
> > Hi
> >
> > Are you able to try with a newer Camel version as we have improved
> > camel-kafka a lot over the last couple of years.
> >
> >
> >
> >
> > On Fri, Jun 24, 2022 at 4:06 PM Rohan Emmanuel <
> rohan.emmanue...@gmail.com
> > >
> > wrote:
> >
> > > hi,
> > > i can using camel 3.7.0 with kafka consumer endpoint with default
> > > configurations. when I start my process, even though i am not sending
> any
> > > messages to the topic, i see high CPU utilization by the java process.
> by
> > > looking at JMeter for the utilization, the KafkaConsumer poll is
> > resulting
> > > in a lot number of calls internally.
> > > Tried different configurations of fetchMinBytes ,reconnectBackoffMs &
> > > reconnectBackoffMaxMs but these did not alter the HIGH CPU utilization.
> > >
> > > Any help on what configuration can be done to avoid high CPU
> utilization?
> > > any pointers would be appreciated.
> > >
> > > --
> > > Regards,
> > > Rohan Emmanuel
> > >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
> >
>
>
> --
> Regards,
> Rohan Emmanuel
>
--
Otavio R. Piske
http://orpiske.net
