Thank you Jonathan! Now I understand the situation better.
I agree with Guozhang about the waiting times for 3.2.0, 3.0.1, and 3.1.1.
Obviously that does not satisfy the organizational requirements you
mentioned.
Best,
Bruno
On 07.02.22 23:24, Guozhang Wang wrote:
Thanks for the clarification Jonathan, I think timing wise, waiting to get
3.2.0 v.s. get 3.1.1 / 3.0.1 would not be much longer.
On Mon, Feb 7, 2022 at 12:16 PM Jonathan Albrecht <jonathan.albre...@ibm.com>
wrote:
Hi Bruno and Guozhang,
Thanks for your thoughtful replies and I hope I can clarify. I work on
a team that helps port open source software to s390x. The example I used
about users needing to be on a specific minor release is a general one
that I see and not specific to kafka and usually its not due to technical
reasons. For example, sometimes organizations want to support the same
version of a component across platforms and back porting support for
s390x mean they can get to a supported version sooner. Maybe in kafka's
case, there are not technical reasons for that to happen but sometimes
there are organizational reasons.
Hope that clarifies where I'm coming from. I definitely don't have a
specific technical issue that would be solved by back porting. I
understand there are risks and it might not be appropriate for a bugfix
release which is why I wanted to ask first before going any further.
Thanks,
Jon
"Bruno Cadonna" <cado...@apache.org> wrote on 2022-02-07 05:30:05 AM:
From: "Bruno Cadonna" <cado...@apache.org>
To: dev@kafka.apache.org
Date: 2022-02-07 05:30 AM
Subject: [EXTERNAL] Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?
Hi Jonathan and Guozhang,
I am pretty sure that upgrading RocksDB in 3.0.x and 3.1.x should not be
an issue, since we recently upgraded it to 6.27.3 on trunk and judging
from the compatibility report in the ticket [1] and the code the API
does not break backward-compatibility for the AK 3.x series.
I am wondering, why users would not be willing to use the upcoming 3.2.0
if they are migrating or creating a Kafka Streams application on the
s390x platform. Even if we ported the upgrade to 3.1.1 and 3.0.1, they
would need to wait until those versions are released. Additionally,
their applications need to comply with 3.x APIs and APIs should be
backward compatible in the 3.x releases. So, everything that works with
3.1.1 and 3.0.1 should also work with 3.2.0. Of course there could be an
issue in 3.2.0 that will force them to use a 3.1.1 or 3.0.1, but that
would be rather an exception that we can then fix when the exception
happens.
Jonathan, could you clarify what the motivation of using hypothetical
3.1.1 or 3.0.1 with the RocksDB upgrade instead of 3.2.0 on a platform
that was not supported before (i.e., 3.1.0 and 3.0.0) might be?
In the end, it is always a risk to upgrade a library in a bugfix release
without a critical issue that the upgrade fixes.
Best,
Bruno
[1]
https://issues.apache.org/jira/browse/KAFKA-13599
On 07.02.22 04:19, Guozhang Wang wrote:
Hi Jonathan,
I'm not against the idea of upgrading in 3.0.x and 3.1.x, assuming that
the
v6.27.3 version does not make any API or any semantic behavioral
changes.
But I can only speak for myself, not the whole community. For older
versions as Bruno mentioned since there's compatibility issues we
cannot
upgrade RocksDB any more.
Guozhang
On Thu, Feb 3, 2022 at 1:56 PM Jonathan Albrecht
<jonathan.albre...@ibm.com>
wrote:
Thanks Guozhang, yes the motivation is to support the s390x platform.
It's
not a critical bug for other platforms.
Any chance that gaining platform support is also a valid reason? I was
hoping it would be but I won't submit a PR if it isn't.
Thanks,
Jon
"Guozhang Wang" <wangg...@gmail.com> wrote on 2022-02-03 02:14:34 PM:
From: "Guozhang Wang" <wangg...@gmail.com>
To: "dev" <dev@kafka.apache.org>
Date: 2022-02-03 02:15 PM
Subject: [EXTERNAL] Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?
Hello Jonathan,
I think Bruno's point is that we can only upgrade in the bugfix
releases if
the old version of rocksDB has a critical bug that the new version
would
fix. For 6.27.3 it seems not fixing a critical bug but for a new
feature,
and hence unless we change the policy we cannot upgrade in 3.0.1 /
3.1.1
releases.
Guozhang
On Thu, Feb 3, 2022 at 9:22 AM Jonathan Albrecht
<jonathan.albre...@ibm.com> wrote:
Thanks for the info Bruno. In that case, if no other concerns, I'll
try
updating RocksDB to v6.27.3 on the 3.0 and 3.1 branches and file
issues and
PRs if everything looks good.
Jon
"Bruno Cadonna" <cado...@apache.org> wrote on 2022-02-03 10:40:00
AM:
From: "Bruno Cadonna" <cado...@apache.org>
To: dev@kafka.apache.org
Date: 2022-02-03 10:40 AM
Subject: [EXTERNAL] Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?
Hi Jonathan,
We had to wait until AK 3.0 to upgrade RocksDB to 6.19.3 due to
source
compatibility issue. More specifically, we expose RocksDB APIs in
Kafka
Streams for configuring RocksDB and those RocksDB APIs changed. So
upgrading RocksDB was actually a compatibility breaking change. We
had
to wait for the major release 3.0.0 to make the upgrade. That
means, if
the policy allows to upgrade dependencies in bugfix releases we can
only
upgrade RocksDB in bugfix releases for 3.1 and 3.0. Upgrading
RocksDB in
earlier releases would break compatibility.
Best,
Bruno
On 03.02.22 15:15, Jonathan Albrecht wrote:
The rocksdbjni dependency has been upgraded to v6.27.3 on trunk
and
wanted to ask if it would be ok to also upgrade it to v6.27.3 on
the 3.1
branch (and possibly earlier branches). I thought I should ask in
case
there are some policies around changing dependency versions in
point
releases.
The motivation is that this is the first version of rocksdbjni
that
supports s390x and it allows kafka to be built out of the box on
this
platform. Having this support in earlier releases helps users on
s390x
that
may need a specific minor release.
If upgrading earlier releases is ok, how far back would be
reasonable?
I'm
happy to create the issues and PRs and do the local testing, of
course.
Thanks,
Jonathan Albrecht
Advisory Software Developer
Linux on IBM Z Open Source Ecosystem
1 905 413 3577 Office
jonathan.albre...@ibm.com
IBM
--
-- Guozhang
