Hi Jonathan and Guozhang,
I am pretty sure that upgrading RocksDB in 3.0.x and 3.1.x should not be
an issue, since we recently upgraded it to 6.27.3 on trunk and judging
from the compatibility report in the ticket [1] and the code the API
does not break backward-compatibility for the AK 3.x series.
I am wondering, why users would not be willing to use the upcoming 3.2.0
if they are migrating or creating a Kafka Streams application on the
s390x platform. Even if we ported the upgrade to 3.1.1 and 3.0.1, they
would need to wait until those versions are released. Additionally,
their applications need to comply with 3.x APIs and APIs should be
backward compatible in the 3.x releases. So, everything that works with
3.1.1 and 3.0.1 should also work with 3.2.0. Of course there could be an
issue in 3.2.0 that will force them to use a 3.1.1 or 3.0.1, but that
would be rather an exception that we can then fix when the exception
happens.
Jonathan, could you clarify what the motivation of using hypothetical
3.1.1 or 3.0.1 with the RocksDB upgrade instead of 3.2.0 on a platform
that was not supported before (i.e., 3.1.0 and 3.0.0) might be?
In the end, it is always a risk to upgrade a library in a bugfix release
without a critical issue that the upgrade fixes.
Best,
Bruno
[1] https://issues.apache.org/jira/browse/KAFKA-13599
On 07.02.22 04:19, Guozhang Wang wrote:
Hi Jonathan,
I'm not against the idea of upgrading in 3.0.x and 3.1.x, assuming that the
v6.27.3 version does not make any API or any semantic behavioral changes.
But I can only speak for myself, not the whole community. For older
versions as Bruno mentioned since there's compatibility issues we cannot
upgrade RocksDB any more.
Guozhang
On Thu, Feb 3, 2022 at 1:56 PM Jonathan Albrecht <jonathan.albre...@ibm.com>
wrote:
Thanks Guozhang, yes the motivation is to support the s390x platform. It's
not a critical bug for other platforms.
Any chance that gaining platform support is also a valid reason? I was
hoping it would be but I won't submit a PR if it isn't.
Thanks,
Jon
"Guozhang Wang" <wangg...@gmail.com> wrote on 2022-02-03 02:14:34 PM:
From: "Guozhang Wang" <wangg...@gmail.com>
To: "dev" <dev@kafka.apache.org>
Date: 2022-02-03 02:15 PM
Subject: [EXTERNAL] Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?
Hello Jonathan,
I think Bruno's point is that we can only upgrade in the bugfix releases
if
the old version of rocksDB has a critical bug that the new version would
fix. For 6.27.3 it seems not fixing a critical bug but for a new feature,
and hence unless we change the policy we cannot upgrade in 3.0.1 / 3.1.1
releases.
Guozhang
On Thu, Feb 3, 2022 at 9:22 AM Jonathan Albrecht
<jonathan.albre...@ibm.com>
wrote:
Thanks for the info Bruno. In that case, if no other concerns, I'll try
updating RocksDB to v6.27.3 on the 3.0 and 3.1 branches and file issues
and
PRs if everything looks good.
Jon
"Bruno Cadonna" <cado...@apache.org> wrote on 2022-02-03 10:40:00 AM:
From: "Bruno Cadonna" <cado...@apache.org>
To: dev@kafka.apache.org
Date: 2022-02-03 10:40 AM
Subject: [EXTERNAL] Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?
Hi Jonathan,
We had to wait until AK 3.0 to upgrade RocksDB to 6.19.3 due to
source
compatibility issue. More specifically, we expose RocksDB APIs in
Kafka
Streams for configuring RocksDB and those RocksDB APIs changed. So
upgrading RocksDB was actually a compatibility breaking change. We
had
to wait for the major release 3.0.0 to make the upgrade. That means,
if
the policy allows to upgrade dependencies in bugfix releases we can
only
upgrade RocksDB in bugfix releases for 3.1 and 3.0. Upgrading RocksDB
in
earlier releases would break compatibility.
Best,
Bruno
On 03.02.22 15:15, Jonathan Albrecht wrote:
The rocksdbjni dependency has been upgraded to v6.27.3 on trunk and
I
wanted to ask if it would be ok to also upgrade it to v6.27.3 on
the
3.1
branch (and possibly earlier branches). I thought I should ask in
case
there are some policies around changing dependency versions in
point
releases.
The motivation is that this is the first version of rocksdbjni that
supports s390x and it allows kafka to be built out of the box on
this
platform. Having this support in earlier releases helps users on
s390x
that
may need a specific minor release.
If upgrading earlier releases is ok, how far back would be
reasonable?
I'm
happy to create the issues and PRs and do the local testing, of
course.
Thanks,
Jonathan Albrecht
Advisory Software Developer
Linux on IBM Z Open Source Ecosystem
1 905 413 3577 Office
jonathan.albre...@ibm.com
IBM
--
-- Guozhang
