Hi all, I prepared a quick fix: https://github.com/apache/pulsar/pull/24195 so that we can fix the regression quickly in 3.0, 3.3 and 4.0 releases. I devoted my time addressing this breaking change because it's my responsibility that I missed the previous open discussion in the ML list. But I didn't have more time to write a PIP and go through the PIP process for that.
Thanks, Yunze On Tue, Apr 22, 2025 at 11:24 PM Yunze Xu <x...@apache.org> wrote: > > Hi Zixuan, > > If you're maintaining your own private KoP fork and upgrading the > Pulsar dependency to include this PR, all tests in > `GroupCoordinatorTest` fail without changes. I didn't have a chance to > look into all the failed tests for now. But at least, there are 37 > tests failing due to the same reason. I've verified that some can be > fixed by creating a partitioned topic manually. > > Even out of the scope of KoP, Pulsar's partitioned topic can be used > as a few partitions and a metadata that represents the partition > count. It has been valid and allowed to create producers or readers > directly on partitions if the partition count is known in advance > (e.g. via the admin API or UI). > - producer0 on `topic-partition-0` > - producer1 on `topic-partition-1` > - ... > > There is a trick that this partitioned topic can be deleted by force > and then producers will reconnect the partitions to continue > producing. However, with this change, the producers will fail to > reconnect due to `NotAllowedException`, Sometimes, users just want to > discard huge legacy data. Regarding the loss of partition metadata, it > can be recreated by creating the same partitioned topic again. I admit > it's a tricky solution. > > Thanks, > Yunze > > > > > On Tue, Apr 22, 2025 at 10:33 PM Zixuan Liu <zix...@apache.org> wrote: > > > > I'm aware that #24118 breaks some test cases, but I believe those test > > cases were incorrect. I also flagged this PR as introducing breaking > > changes in the mailing list, please see > > https://lists.apache.org/thread/g1jq0vkq0wt28mdnbg63bpw8fpx9ml42 > > > > I'd like to better understand the scope of the breakage. Is it limited to > > internal KoP usage—such as test cases where developers use > > partitioned-topic format without metadata? Or does it also affect users who > > might be doing this in practice? In our organization, we also maintain KoP > > and have updated our test cases accordingly. > > > > Personally, I believe that the partitioned topic should have associated > > metadata. If someone uses the partitioned format without metadata, it feels > > more like a workaround or an unsupported edge case. > > > > I'm open to discussion. If this behavior is something users actually depend > > on, then we should talk about how to handle it more gracefully. > > > > Thanks, > > Zixuan
