Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-13 Thread David Arthur
Stan, I think that makes sense. I'll update the KIP and start the vote shortly. On Thu, Aug 8, 2019 at 12:54 PM Stanislav Kozlovski wrote: > What do people think if we exposed: > * eligible topics/replicas pending delete > * ineligible topics/replicas pending delete > > On Thu, Aug 8, 2019 at 5:

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-08 Thread Stanislav Kozlovski
What do people think if we exposed: * eligible topics/replicas pending delete * ineligible topics/replicas pending delete On Thu, Aug 8, 2019 at 5:16 PM David Arthur wrote: > It looks like topicsIneligibleForDeletion is a subset of topicsToBeDeleted > in the controller. > > On Thu, Aug 8, 2019 a

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-08 Thread David Arthur
It looks like topicsIneligibleForDeletion is a subset of topicsToBeDeleted in the controller. On Thu, Aug 8, 2019 at 11:16 AM Stanislav Kozlovski wrote: > ineligible replicas/topics are not included in the pending metrics, right? > If so, sounds good to me. > > On Thu, Aug 8, 2019 at 4:12 PM Dav

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-08 Thread Stanislav Kozlovski
ineligible replicas/topics are not included in the pending metrics, right? If so, sounds good to me. On Thu, Aug 8, 2019 at 4:12 PM David Arthur wrote: > Yes I think exposing ineligible topics would be useful as well. The > controller also tracks this ineligible state for replicas. Would that be

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-08 Thread David Arthur
Yes I think exposing ineligible topics would be useful as well. The controller also tracks this ineligible state for replicas. Would that be useful to expose as well? In that case, we'd be up to four new metrics: * topics pending delete * replicas pending delete * ineligible topics * ineligible re

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-07 Thread Jason Gustafson
Thanks for the KIP. This is useful. The controller also maintains a set for topics which are awaiting deletion, but currently ineligible. A topic which is undergoing reassignment, for example, is ineligible for deletion. Would it make sense to have a metric for this as well? -Jason On Wed, Aug 7,

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-07 Thread David Arthur
Updated the KIP with a count of replicas awaiting deletion. On Wed, Aug 7, 2019 at 9:37 AM David Arthur wrote: > Thanks for the feedback, Stan. That's a good point about the partition > count -- I'll poke around and see if I can surface this value in the > Controller. > > On Tue, Aug 6, 2019 at

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-07 Thread David Arthur
Thanks for the feedback, Stan. That's a good point about the partition count -- I'll poke around and see if I can surface this value in the Controller. On Tue, Aug 6, 2019 at 8:13 AM Stanislav Kozlovski wrote: > Thanks for the KIP David, > > As you mentioned in the KIP - "when a large number of

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-06 Thread Stanislav Kozlovski
Thanks for the KIP David, As you mentioned in the KIP - "when a large number of topics (partitions, really) are deleted at once, it can take significant time for the Controller to process everything. In that sense, does it make sense to have the metric expose the number of partitions that are pend

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-05 Thread Harsha Chintalapani
Thanks for the KIP. Its useful metric to have. LGTM. -Harsha On Mon, Aug 05, 2019 at 11:24 AM, David Arthur wrote: > Hello all, I'd like to start a discussion for > https://cwiki.apache.org/confluence/display/KAFKA/ > KIP-503%3A+Add+metric+for+number+of+topics+marked+for+deletion > > Thanks!