Re: [DISCUSS] KIP-724: Drop support for message formats v0 and v1

2021-06-09 Thread Jun Rao
Hi, Ismael, Sounds good. The KIP looks good to me. Thanks, Jun On Wed, Jun 9, 2021 at 9:16 AM Ismael Juma wrote: > Thanks Jun. After further thought and discussion with Jason, we decided to > move `log.record.version.force.upgrade` to a "Future Work" section. > > There are more details in the

Re: [DISCUSS] KIP-724: Drop support for message formats v0 and v1

2021-06-09 Thread Ismael Juma
Thanks Jun. After further thought and discussion with Jason, we decided to move `log.record.version.force.upgrade` to a "Future Work" section. There are more details in the KIP, but there are some complexities in making it possible to remove read support for v0/v1 formats. Given that, we decided t

Re: [DISCUSS] KIP-724: Drop support for message formats v0 and v1

2021-06-08 Thread Jun Rao
Hi, Ismael, Thanks for the KIP. Regarding log.record.version.force.upgrade: 1. Is that a topic level or a broker level config? The slight benefit of topic level is that it allows people to upgrade incrementally. The potential downside is the additional complexity associated with it. 2. When log.r

Re: [DISCUSS] KIP-724: Drop support for message formats v0 and v1

2021-06-08 Thread Ismael Juma
Hi Jason, Thanks for the thoughtful comments. Responses below. 1. Good point. I updated the KIP to up-convert during replication if the IBP is 3.0. I am still not 100% sure if this is truly required, but it does seem like a good idea to avoid flip-flopping like this. I'll follow up with you to un

Re: [DISCUSS] KIP-724: Drop support for message formats v0 and v1

2021-06-04 Thread Jason Gustafson
Hi Ismael, I agree it would be awesome to drop support for the old formats! A few comments below: 1. Suppose that a partition with 3 replicas begins at v0. One broker is upgraded to v2 with the new "force upgrade" config as part of a rolling restart, which leaves two replicas on v0 and one on v2.

Re: [DISCUSS] KIP-724: Drop support for message formats v0 and v1

2021-06-04 Thread Ismael Juma
If there are no comments or objections, I will start a vote on this soon. On Tue, Jun 1, 2021 at 7:59 AM Ismael Juma wrote: > Hi all, > > It's time to start the process of sunsetting message formats v0 and v1 in > order to establish a new baseline in terms of supported client/broker > behavior a

[DISCUSS] KIP-724: Drop support for message formats v0 and v1

2021-06-01 Thread Ismael Juma
Hi all, It's time to start the process of sunsetting message formats v0 and v1 in order to establish a new baseline in terms of supported client/broker behavior and to improve maintainability & supportability of Kafka. Please take a look at the proposal: https://cwiki.apache.org/confluence/displa