+1 to either doing full release or deferring to 2.56.0.
Jan
On 3/28/24 16:52, Yi Hu via dev wrote:
> Just releasing Python can break multi-lang by default (unless
expansion service is overridden manually) since we match versions
across languages when picking the default expansion service.
Yes, that's why I proposed "the source code of release candidate (e.g.
apache_beam/version.py) still reads 2.55.0. " Anyways it seems doing a
full release is preferred as it reduces the risk of breakages.
On Thu, Mar 28, 2024 at 11:38 AM Chamikara Jayalath via dev
<dev@beam.apache.org> wrote:
On Thu, Mar 28, 2024 at 8:36 AM Chamikara Jayalath
<chamik...@google.com> wrote:
Just releasing Python can break multi-lang by default (unless
expansion service is overridden manually) since we match
versions across languages when picking the default expansion
service.
https://github.com/apache/beam/blob/2f8854a3e34f31c1cc034f95ad36f317abc906ff/sdks/python/apache_beam/utils/subprocess_server.py#L42
Correct link:
https://github.com/apache/beam/blob/2f8854a3e34f31c1cc034f95ad36f317abc906ff/sdks/python/apache_beam/utils/subprocess_server.py#L352
Thanks,
Cham
On Thu, Mar 28, 2024 at 8:26 AM Danny McCormick via dev
<dev@beam.apache.org> wrote:
> The patch itself [1] is trivial, however, the release
process is not trivial. There is little documentation nor
practice for a patch release process. I could imagine two
options
I think there's not a ton of documentation because we
haven't done it, but all the release workflows were
authored in such a way that they should "just work",
outside of cutting the release branch itself. So the
workflow should be almost identical to the existing one,
but with several steps skipped (cherry picks, beam
website, most validation). Notably, this shouldn't be any
easier/harder if we're doing it for one language or all 3.
I can take that on if needed.
> Besides, there should be a Beam YAML validation workflow
and added in
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253
> If we do a patch release for Python SDK, let's also
patch another known issue for which fix is available:
https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1
+1 to both of these
On Thu, Mar 28, 2024 at 11:25 AM Yi Hu via dev
<dev@beam.apache.org> wrote:
Thanks Valentyn for raising this. In this case, Python
containers will also be included. Different from PyPI
wheels, docker tag can override so it can stay with 2.55.0
On Thu, Mar 28, 2024 at 11:15 AM Valentyn Tymofieiev
<valen...@google.com> wrote:
If we do a patch release for Python SDK, let's
also patch another known issue for which fix is
available:
https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1
On Thu, Mar 28, 2024 at 8:01 AM Yi Hu via dev
<dev@beam.apache.org> wrote:
2.55.0 release manager here
The patch itself [1] is trivial, however, the
release process is not trivial. There is
little documentation nor practice for a patch
release process. I could imagine two options
1. Do a full "2.55.1" release
2. Do a patch release only for Python SDK, that is
a. cherry-pick [1] into release-2.55.0 branch
b. tag a 2.55.1rc1 release candidate - note
that the source code of release candidate
(e.g. apache_beam/version.py) still reads
2.55.0. This ensures Python SDK picks up the
Java expansion service / job server of
existing version (2.55.0). We did it once for
Go SDK
(https://github.com/apache/beam/tree/sdks/v2.48.2)
c. Build the release candidate for Python
wheels (also Python containers? Not sure if it
is needed)
d. send out the RC for validation
e. finalize the release
If we decided to do a patch release I would
prefer option 2. I can take on that if decided
to do. However, if we decide do a full release
(or both Java and Python) I would suggest
defer to next release cycle, as the release
process itself could take ~10 days minimum if
there is single RC.
Besides, there should be a Beam YAML
validation workflow and added in
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253
[1] https://github.com/apache/beam/pull/30780
On Thu, Mar 28, 2024 at 10:22 AM Danny
McCormick via dev <dev@beam.apache.org> wrote:
+1 on a patch release - we've done a fair
amount of work to make releasing easier,
and one of my hopes is that it will enable
quick patches like this. I'd vote we try
to fix the underlying Java piece as well,
though, doing a patch release for one
language shouldn't be significantly
cheaper than doing it for multiple languages.
Thanks,
Danny
On Wed, Mar 27, 2024 at 7:19 PM Robert
Burke <rob...@frantil.com> wrote:
+1 to a targeted patch release.
We did the same for the Go SDK a
little while back. It would be good to
see what's different for a different SDK.
On Wed, Mar 27, 2024, 4:01 PM Robert
Bradshaw via dev <dev@beam.apache.org>
wrote:
Given the severity of
the breakage, and the simplicity
of the workaround, I'm in favor of
a patch release. I think we could
do Python-only, which would make
the process even more lightweight.
On Wed, Mar 27, 2024 at 3:48 PM
Jeff Kinard <j...@thekinards.com>
wrote:
Hi all,
Beam 2.55 was released with a
bug that causes WriteToJson on
Beam YAML to fail when using
the Java variant. This also
affects any user attempting to
use the Xlang
JsonWriteTransformProvider -
https://github.com/apache/beam/blob/master/sdks/java/io/json/src/main/java/org/apache/beam/sdk/io/json/providers/JsonWriteTransformProvider.java
This is due to a change to
https://github.com/apache/beam/blob/master/sdks/java/io/json/build.gradle
that removed
a dependency on everit which
also removed it from being
packaged into the expansion
service JAR:
beam-sdks-java-extensions-sql-expansion-service-2.55.0.jar
There is a temporary fix to
disable the provider in Beam
YAML:
https://github.com/apache/beam/pull/30777
I think with the total loss of
function, and a trivial fix,
it is worth creating a patch
release of Beam 2.55 to
include this fix.
- Jeff
