Hi Bruno and Guozhang,
Thanks for your thoughtful replies and I hope I can clarify. I work on
a team that helps port open source software to s390x. The example I used
about users needing to be on a specific minor release is a general one
that I see and not specific to kafka and usually its not due to technical
reasons. For example, sometimes organizations want to support the same
version of a component across platforms and back porting support for
s390x mean they can get to a supported version sooner. Maybe in kafka's
case, there are not technical reasons for that to happen but sometimes
there are organizational reasons.
Hope that clarifies where I'm coming from. I definitely don't have a
specific technical issue that would be solved by back porting. I
understand there are risks and it might not be appropriate for a bugfix
release which is why I wanted to ask first before going any further.
Thanks,
Jon
"Bruno Cadonna" <cado...@apache.org> wrote on 2022-02-07 05:30:05 AM:
> From: "Bruno Cadonna" <cado...@apache.org>
> To: dev@kafka.apache.org
> Date: 2022-02-07 05:30 AM
> Subject: [EXTERNAL] Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?
>
> Hi Jonathan and Guozhang,
>
> I am pretty sure that upgrading RocksDB in 3.0.x and 3.1.x should not be
> an issue, since we recently upgraded it to 6.27.3 on trunk and judging
> from the compatibility report in the ticket [1] and the code the API
> does not break backward-compatibility for the AK 3.x series.
>
> I am wondering, why users would not be willing to use the upcoming 3.2.0
> if they are migrating or creating a Kafka Streams application on the
> s390x platform. Even if we ported the upgrade to 3.1.1 and 3.0.1, they
> would need to wait until those versions are released. Additionally,
> their applications need to comply with 3.x APIs and APIs should be
> backward compatible in the 3.x releases. So, everything that works with
> 3.1.1 and 3.0.1 should also work with 3.2.0. Of course there could be an
> issue in 3.2.0 that will force them to use a 3.1.1 or 3.0.1, but that
> would be rather an exception that we can then fix when the exception
> happens.
>
> Jonathan, could you clarify what the motivation of using hypothetical
> 3.1.1 or 3.0.1 with the RocksDB upgrade instead of 3.2.0 on a platform
> that was not supported before (i.e., 3.1.0 and 3.0.0) might be?
>
> In the end, it is always a risk to upgrade a library in a bugfix release
> without a critical issue that the upgrade fixes.
>
>
> Best,
> Bruno
>
>
> [1]
https://issues.apache.org/jira/browse/KAFKA-13599
>
> On 07.02.22 04:19, Guozhang Wang wrote:
> > Hi Jonathan,
> >
> > I'm not against the idea of upgrading in 3.0.x and 3.1.x, assuming that
the
> > v6.27.3 version does not make any API or any semantic behavioral
changes.
> > But I can only speak for myself, not the whole community. For older
> > versions as Bruno mentioned since there's compatibility issues we
cannot
> > upgrade RocksDB any more.
> >
> >
> > Guozhang
> >
> > On Thu, Feb 3, 2022 at 1:56 PM Jonathan Albrecht
<jonathan.albre...@ibm.com>
> > wrote:
> >
> >>
> >>
> >> Thanks Guozhang, yes the motivation is to support the s390x platform.
It's
> >> not a critical bug for other platforms.
> >>
> >> Any chance that gaining platform support is also a valid reason? I was
> >> hoping it would be but I won't submit a PR if it isn't.
> >>
> >> Thanks,
> >>
> >> Jon
> >>
> >> "Guozhang Wang" <wangg...@gmail.com> wrote on 2022-02-03 02:14:34 PM:
> >>
> >>> From: "Guozhang Wang" <wangg...@gmail.com>
> >>> To: "dev" <dev@kafka.apache.org>
> >>> Date: 2022-02-03 02:15 PM
> >>> Subject: [EXTERNAL] Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?
> >>>
> >>> Hello Jonathan,
> >>>
> >>> I think Bruno's point is that we can only upgrade in the bugfix
releases if
> >>> the old version of rocksDB has a critical bug that the new version
would
> >>> fix. For 6.27.3 it seems not fixing a critical bug but for a new
feature,
> >>> and hence unless we change the policy we cannot upgrade in 3.0.1 /
3.1.1
> >>> releases.
> >>>
> >>>
> >>> Guozhang
> >>>
> >>> On Thu, Feb 3, 2022 at 9:22 AM Jonathan Albrecht
<jonathan.albre...@ibm.com> wrote:
> >>>
> >>>>
> >>>>
> >>>>
> >>>> Thanks for the info Bruno. In that case, if no other concerns, I'll
try
> >>>> updating RocksDB to v6.27.3 on the 3.0 and 3.1 branches and file
issues and
> >>>> PRs if everything looks good.
> >>>>
> >>>> Jon
> >>>>
> >>>>
> >>>> "Bruno Cadonna" <cado...@apache.org> wrote on 2022-02-03 10:40:00
AM:
> >>>>
> >>>>> From: "Bruno Cadonna" <cado...@apache.org>
> >>>>> To: dev@kafka.apache.org
> >>>>> Date: 2022-02-03 10:40 AM
> >>>>> Subject: [EXTERNAL] Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?
> >>>>>
> >>>>> Hi Jonathan,
> >>>>>
> >>>>> We had to wait until AK 3.0 to upgrade RocksDB to 6.19.3 due to
source
> >>>>> compatibility issue. More specifically, we expose RocksDB APIs in
Kafka
> >>>>> Streams for configuring RocksDB and those RocksDB APIs changed. So
> >>>>> upgrading RocksDB was actually a compatibility breaking change. We
had
> >>>>> to wait for the major release 3.0.0 to make the upgrade. That
means, if
> >>>>> the policy allows to upgrade dependencies in bugfix releases we can
only
> >>>>> upgrade RocksDB in bugfix releases for 3.1 and 3.0. Upgrading
RocksDB in
> >>>>> earlier releases would break compatibility.
> >>>>>
> >>>>> Best,
> >>>>> Bruno
> >>>>>
> >>>>> On 03.02.22 15:15, Jonathan Albrecht wrote:
> >>>>>>
> >>>>>>
> >>>>>> The rocksdbjni dependency has been upgraded to v6.27.3 on trunk
and
> >>>>>> wanted to ask if it would be ok to also upgrade it to v6.27.3 on
the 3.1
> >>>>>> branch (and possibly earlier branches). I thought I should ask in
case
> >>>>>> there are some policies around changing dependency versions in
point
> >>>>>> releases.
> >>>>>>
> >>>>>> The motivation is that this is the first version of rocksdbjni
that
> >>>>>> supports s390x and it allows kafka to be built out of the box on
this
> >>>>>> platform. Having this support in earlier releases helps users on
s390x
> >>>>>> that
> >>>>>> may need a specific minor release.
> >>>>>>
> >>>>>> If upgrading earlier releases is ok, how far back would be
reasonable?
> >>>>>> I'm
> >>>>>> happy to create the issues and PRs and do the local testing, of
course.
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Jonathan Albrecht
> >>>>>> Advisory Software Developer
> >>>>>> Linux on IBM Z Open Source Ecosystem
> >>>>>> 1 905 413 3577 Office
> >>>>>> jonathan.albre...@ibm.com
> >>>>>>
> >>>>>> IBM
> >>>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> -- Guozhang
> >>
> >
> >
