Hi All,

Since KIP-455 is passed, I would like to revive this proposal. I have
reduced the scope so that it is just changing the way we compute URP and
adding a new metric for the number of reassigning partitions. Please take a
look:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-352%3A+Distinguish+URPs+caused+by+reassignment
.

Thanks,
Jason

On Sun, Aug 5, 2018 at 3:41 PM Dong Lin <lindon...@gmail.com> wrote:

> Hey Jason,
>
> Yes, I also think the right solution is probably to include the
> expected_replication_factor in the per-topic znode and include this
> information in the UpdateMetadataRequest.
>
> And I agree with Gwen and Ismael that it is reasonable to redefine URP as
> those partitions whose ISR set size < expected replication factor. Given
> that URP is being used for monitoring cluster availability and and
> reassignment progress, it seems that one of them will break depending on
> the URP definition. It seems better to keep the legitimate use-case, i.e.
> monitoring cluster availability. Users who want to monitor reassignment
> progress probably should use the "kafka-reassign-partitions.sh --verify".
>
>
> Thanks,
> Dong
>
>
>
> On Fri, Aug 3, 2018 at 2:57 PM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > @Dong
> >
> > Ugh, you are right. This is indeed trickier than I imagined. You could
> > argue that the problem here is that there is no single source of truth
> for
> > the expected replication factor. While a reassignment is in progress, the
> > reassignment itself has the expected replica set. Otherwise, it is stored
> > partition metadata itself. This is why manually deleting the reassignment
> > is problematic. We lose the expected replica set and we depend on users
> to
> > reinstall it. I guess I'm starting to think that the way we track
> > reassignment state in the controller is problematic. In addition to the
> > problems caused by deletion, we cannot easily change an existing
> > reassignment.
> >
> > High level what I'm thinking is that we need to move the pending
> > reassignment state out of the single znode and into the individual
> metadata
> > of the reassigned partitions so that there is a single source of truth
> for
> > the expected replicas (and hence the replication factor). This would also
> > give us an easier mechanism to manage the batching of multiple
> > reassignments. Let me think on it a bit and see if I can come up with a
> > proposal.
> >
> > @Gwen, @Ismael
> >
> > That is fair. I also prefer to redefine URP if we think the compatibility
> > impact is a lower concern than continued misuse.
> >
> > -Jason
> >
> >
> >
> > On Fri, Aug 3, 2018 at 12:25 PM, Ismael Juma <ism...@juma.me.uk> wrote:
> >
> > > RIght, that was my thinking too.
> > >
> > > Ismael
> > >
> > > On Fri, Aug 3, 2018 at 12:04 PM Gwen Shapira <g...@confluent.io>
> wrote:
> > >
> > > > On Fri, Aug 3, 2018 at 11:23 AM, Jason Gustafson <ja...@confluent.io
> >
> > > > wrote:
> > > >
> > > > > Hey Ismael,
> > > > >
> > > > > Yeah, my initial inclination was to redefine URP as well. My only
> > doubt
> > > > was
> > > > > how it would affect existing tools which might depend on URPs to
> > track
> > > > the
> > > > > progress of a reassignment. I decided to be conservative in the
> end,
> > > but
> > > > > I'd reconsider if we think it is not a major concern. It is
> annoying
> > to
> > > > > need a new category.
> > > > >
> > > >
> > > > There are existing tools that use URP to track reassignment, but
> there
> > > are
> > > > many more tools that use URP for monitoring and alerting. If I
> > understand
> > > > Ismael's suggestion correctly, a re-definition will improve the
> > > reliability
> > > > of the monitoring tools (since there won't be false alerts in case of
> > > > re-assignment) without having to switch to a new metric.
> > > >
> > > > I think we should choose the proposal that improves the more common
> > usage
> > > > of the metric, in this case, failure monitoring rather than
> > reassignment.
> > > >
> > > >
> > > > >
> > > > > About your question about storage in ZK, I can't think of anything
> > > > > additional that we need. Probably the main difficulty is getting
> > access
> > > > to
> > > > > the replication factor in the topic utility. My basic thought was
> > just
> > > to
> > > > > collect the URPs (as we know them today) and use the config API to
> > > > > partition them based on the replication factor. Do you see any
> > problems
> > > > > with this?
> > > > >
> > > > > -Jason
> > > > >
> > > > >
> > > > > On Thu, Aug 2, 2018 at 12:14 PM, Ismael Juma <ism...@juma.me.uk>
> > > wrote:
> > > > >
> > > > > > Thanks Jason. This is definitely a pain point. I actually prefer
> > the
> > > > > option
> > > > > > to redefine what under-replicated means (currently under rejected
> > > > > > alternatives). Also, do we need to make changes to what we store
> in
> > > ZK?
> > > > > If
> > > > > > so, that should be in the KIP too.
> > > > > >
> > > > > > Ismael
> > > > > >
> > > > > > On Thu, Aug 2, 2018 at 11:45 AM Jason Gustafson <
> > ja...@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey All,
> > > > > > >
> > > > > > > Another day, another KIP. This one is hopefully
> straightforward:
> > > > > > >
> > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-352%
> > > > > > 3A+Distinguish+URPs+caused+by+reassignment
> > > > > > > .
> > > > > > > Have a look and let me know what you think!
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Jason
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *Gwen Shapira*
> > > > Product Manager | Confluent
> > > > 650.450.2760 | @gwenshap
> > > > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > > > <http://www.confluent.io/blog>
> > > >
> > >
> >
>

Reply via email to