As described in
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Network
the Kafka protocol is message-oriented, thus largely connectionless (TCP is
used for reliability, not for connectedness). The overhead is just the TCP
handshake - if the TCP handshake is your largest performance problem, then
I have to congratulate you :-).

On Tue, Mar 22, 2016 at 1:02 AM, BYEONG-GI KIM <bg...@bluedigm.com> wrote:

> I have not measured that value exactly, but I just thought the case when we
> use db; the frequent connection/disconnection causes overhead so that
> connection pool is generally used to solve the problem.
>
> Does the kafka bring less overhead comparing with that case as much as what
> the connection pool doesn't need to be considered?
>
> Best regards
>
> bgkim
>
> 2016-03-22 11:40 GMT+09:00 Christian Posta <christian.po...@gmail.com>:
>
> > Can you explain more? Have you measured the overhead of opening the
> > connections?
> >
> > If I'm not mistaken, Kafka manages the connections under the covers to
> each
> > of the brokers that have topics (leader partitions) from which you're
> > consuming. The connection(s) to each partition leader will stay around as
> > long as you're consuming from them (ie, calling poll()) method. If you
> > aren't polling the broker for messages within the session timeout period,
> > the heartbeat mechanism will actually kick the consumer out and do a
> > rebalance. I'd be curious to understand more about your usecase to see
> > how/when you're experiencing overhead of the connection handshaking.
> >
> > On Mon, Mar 21, 2016 at 7:33 PM, BYEONG-GI KIM <bg...@bluedigm.com>
> wrote:
> >
> > > Hello. I have a question that the latest kafka, 0.9.0.1, provides any
> > APIs
> > > for managing connection pool of kafka on both consumer and producer
> > sides.
> > >
> > > I think the overhead which happens while establishing connection from
> > > consumer/producer to kafka broker(s) seems a little heavy.
> > >
> > > Thanks in advance!
> > >
> > > Best regards
> > >
> > > bgkim
> > >
> >
> >
> >
> > --
> > *Christian Posta*
> > twitter: @christianposta
> > http://www.christianposta.com/blog
> > http://fabric8.io
> >
>



-- 

*Cees de Groot*
PRINCIPAL SOFTWARE ENGINEER
[image: PagerDuty logo] <http://pagerduty.com/>
pagerduty.com
c...@pagerduty.com <m...@pagerduty.com>
+1(416)435-4085

[image: Twitter] <http://twitter.com/pagerduty>[image: FaceBook]
<https://www.facebook.com/PagerDuty>[image: Google+]
<https://plus.google.com/114339089137644062989>[image: LinkedIn]
<https://www.linkedin.com/company/pagerduty>[image: Blog]
<https://blog.pagerduty.com/>

Reply via email to