Thanks Colin. I have updated the KIP.
On Tue, Jan 15, 2019 at 9:20 PM Colin McCabe <cmcc...@apache.org> wrote:
> On Tue, Jan 15, 2019, at 12:59, Rajini Sivaram wrote:
> > Hi Colin,
> >
> > `AcceptorIdlePercent` indicates the total amount of time the acceptor is
> > inactive and not accepting any connections because it is blocked on
> > Processors. But I agree the name could be improved. There is back
> pressure
> > at the Java level (which we can't monitor) and back pressure we apply
> with
> > blocking queues which the metric refers to. Perhaps
> > `AcceptorBlockedPercent`reflects
> > that it is the time that the acceptor is blocked?
>
> Yeah, I like "AcceptorBlockedPercent" better.
>
> >
> > As Ismael said, fairness refers to the distribution of time between
> > processing of new connections and processing of existing connections.
>
> OK.
>
> best,
> Colin
>
> >
> > Thanks,
> >
> > Rajini
> >
> >
> > On Tue, Jan 15, 2019 at 7:56 PM Ismael Juma <ism...@juma.me.uk> wrote:
> >
> > > I think the point is that we distribute the time more fairly between
> > > connection handling and other operations where before we could block
> on the
> > > TLS handshake for a long time given a large number of connections.
> > >
> > > Ismael
> > >
> > > On Tue, Jan 15, 2019 at 11:39 AM Colin McCabe <cmcc...@apache.org>
> wrote:
> > >
> > > > Hi Rajini,
> > > >
> > > > Thanks for this. The KIP looks really useful.
> > > >
> > > > >
> > > > > A new metric will be added to track the amount of time Acceptor is
> > > > blocked
> > > > > from accepting connections due to backpressure. This will be a
> yammer
> > > > > Meter, consistent with other SocketServer metrics.
> > > > >
> > > > >
> > > >
> > >
> kafka.network:type=Acceptor,name=AcceptorIdlePercent,listener={listenerName}
> > > > >
> > > >
> > > > Hmm. I was a bit confused by this. When the acceptor is not
> accepting
> > > > connections because there are none coming in, does that count as
> idle?
> > > > When the acceptor is not accepting connections because the connect
> rate
> > > is
> > > > being backpressured, does that count as idle? Would it would be more
> > > > intuitive to have a metric titled AcceptorBackpressuredPercent?
> > > >
> > > > Also, I sort of wonder if titling this "Limit incoming connection
> > > > connection rate" or similar would be clearer than "improving
> fairness."
> > > I
> > > > guess it is unfair that a lot of incoming connections can swamp the
> > > network
> > > > threads right now. But limiting the rate of new connections is
> unfair to
> > > > people connecting. Overall the goal seems to be usability, not
> fairness.
> > > >
> > > > best,
> > > > Colin
> > > >
> > > >
> > > >
> > > > On Tue, Jan 15, 2019, at 04:27, Rajini Sivaram wrote:
> > > > > Hi Jan,
> > > > >
> > > > > If the queue of one Processor is full, we move to the next
> Processor
> > > > > immediately without blocking. So as long as the queue of any
> Processor
> > > is
> > > > > not full, we accept the connection immediately. If the queue of all
> > > > > Processors are full, we assign a Processor and block until the
> > > connection
> > > > > can be added. There is currently no timeout for this. The PR is
> here:
> > > > > https://github.com/apache/kafka/pull/6022
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Rajini
> > > > >
> > > > > On Tue, Jan 15, 2019 at 12:02 PM Jan Filipiak <
> > > jan.filip...@trivago.com>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > > > The connection queue for Processors will be changed to
> > > > > > ArrayBlockingQueue with a fixed size of 20. Acceptor will use
> > > > round-robin
> > > > > > allocation to allocate each new connection to the next available
> > > > Processor
> > > > > > to which the connection can be added without blocking. If a
> > > Processor's
> > > > > > queue is full, the next Processor will be chosen. If the
> connection
> > > > queue
> > > > > > on all Processors are full, Acceptor blocks until the connection
> can
> > > be
> > > > > > added to the selected Processor. No new connections will be
> accepted
> > > > during
> > > > > > this period. The amount of time Acceptor is blocked can be
> monitored
> > > > using
> > > > > > the new AcceptorIdlePercent metric.
> > > > > >
> > > > > > So if the queue of one Processor is full, what is the strategy to
> > > move
> > > > > > to the next queue? Are we using offer with a timeout here? How
> else
> > > can
> > > > > > we make sure that a single slow processor will not block the
> entire
> > > > > > processing? I assume we do not allow us to get stuck during put
> as
> > > you
> > > > > > mention that all queues full is a scenario. I think there is
> quite
> > > some
> > > > > > uncertainty here. Is there any code one could check out?
> > > > > >
> > > > > > Best Jan
> > > > > >
> > > > >
> > > >
> > >
> >
>
