We determined that the same issue exists in the 2.45 release, so we are going to continue finalizing the release candidate. Thank you for your patience.
Thanks, Danny On Wed, Mar 8, 2023 at 3:15 PM Reuven Lax <re...@google.com> wrote: > We are trying to reproduce and debug the issue we saw to validate whether > it was a real regression or not. Will update when we know more. > > On Wed, Mar 8, 2023 at 11:31 AM Danny McCormick <dannymccorm...@google.com> > wrote: > >> >> @Reuven Lax <re...@google.com> found a new potential regression in >> BigQuery I/O, so I have paused the release rollout. I had already pushed >> the Python artifacts and Go tags, but not the Java ones. We have since >> temporarily yanked <https://pypi.org/help/#yanked> the Python release >> and deleted the Go tags, they were live for around an hour. The possible >> regression is in Java, so neither of those releases should be affected, but >> x-lang may not work properly because it depends on versioning. I will >> update this thread with next steps when we know more. >> >> Thanks, >> Danny >> On Wed, Mar 8, 2023 at 5:59 AM Jan Lukavský <je...@seznam.cz> wrote: >> >>> +1 (binding) >>> >>> Tested Java SDK with Flink and Spark 3 runner. >>> >>> Thanks, >>> Jan >>> >>> On 3/8/23 01:53, Valentyn Tymofieiev via dev wrote: >>> >>> +1. Verified the composition of Python containers and ran Python >>> pipelines on Dataflow runner v1 and runner v2. >>> >>> On Tue, Mar 7, 2023 at 4:11 PM Ritesh Ghorse via dev < >>> dev@beam.apache.org> wrote: >>> >>>> +1 (non-binding) >>>> Validated Go SDK quickstart on direct and dataflow runner >>>> >>>> On Tue, Mar 7, 2023 at 10:54 AM Alexey Romanenko < >>>> aromanenko....@gmail.com> wrote: >>>> >>>>> +1 (binding) >>>>> >>>>> Tested with https://github.com/Talend/beam-samples/ >>>>> (Java SDK v8/v11/v17, Spark 3.x runner). >>>>> >>>>> --- >>>>> Alexey >>>>> >>>>> On 7 Mar 2023, at 07:38, Ahmet Altay via dev <dev@beam.apache.org> >>>>> wrote: >>>>> >>>>> +1 (binding) - I validated python quickstarts on direct & dataflow >>>>> runners. >>>>> >>>>> Thank you for doing the release! >>>>> >>>>> On Sat, Mar 4, 2023 at 8:01 AM Chamikara Jayalath via dev < >>>>> dev@beam.apache.org> wrote: >>>>> >>>>>> +1 (binding) >>>>>> >>>>>> Validated multi-language Java and Python pipelines. >>>>>> >>>>>> On Fri, Mar 3, 2023 at 1:59 PM Danny McCormick via dev < >>>>>> dev@beam.apache.org> wrote: >>>>>> >>>>>>> > I have encountered a failure in a Python pipeline running with >>>>>>> Runner v1: >>>>>>> >>>>>>> > RuntimeError: Beam SDK base version 2.46.0 does not match Dataflow >>>>>>> Python worker version 2.45.0. Please check Dataflow worker startup logs >>>>>>> and >>>>>>> make sure that correct version of Beam SDK is installed. >>>>>>> >>>>>>> > We should understand why Python ValidatesRunner tests (which have >>>>>>> passed) didn't catch this error. >>>>>>> >>>>>>> > This can be remediated in Dataflow containers without changes to >>>>>>> the release candidate. >>>>>>> >>>>>>> Good catch! I've kicked off a release to fix this, it should be done >>>>>>> later this evening - I won't be available when it completes, but I would >>>>>>> expect it to be around 5:00 PST. >>>>>>> >>>>>>> On Fri, Mar 3, 2023 at 3:49 PM Danny McCormick < >>>>>>> dannymccorm...@google.com> wrote: >>>>>>> >>>>>>>> Hey Reuven, could you provide some more context on the bug/why it >>>>>>>> is important? Does it meet the standard in >>>>>>>> https://beam.apache.org/contribute/release-guide/#7-triage-release-blocking-issues-in-github? >>>>>>>> >>>>>>>> >>>>>>>> The release branch was cut last Wednesday, so that is why it is not >>>>>>>> included. >>>>>>>> >>>>>>> >>>>>> Seems like this was a revert of a previous commit that was also not >>>>>> included in the 2.46.0 release branch ( >>>>>> https://github.com/apache/beam/pull/25627) ? >>>>>> >>>>>> If so we might not need a new RC but good to confirm. >>>>>> >>>>>> Thanks, >>>>>> Cham >>>>>> >>>>>> >>>>>>>> On Fri, Mar 3, 2023 at 3:24 PM Reuven Lax <re...@google.com> wrote: >>>>>>>> >>>>>>>>> If possible, I would like to see if we could include >>>>>>>>> https://github.com/apache/beam/pull/25642 as we believe this bug >>>>>>>>> has been impacting multiple users. This was merged 4 days ago, but >>>>>>>>> this RC >>>>>>>>> cut does not seem to include it. >>>>>>>>> >>>>>>>>> On Fri, Mar 3, 2023 at 12:18 PM Valentyn Tymofieiev via dev < >>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>> >>>>>>>>>> I have encountered a failure in a Python pipeline running with >>>>>>>>>> Runner v1: >>>>>>>>>> >>>>>>>>>> RuntimeError: Beam SDK base version 2.46.0 does not match >>>>>>>>>> Dataflow Python worker version 2.45.0. Please check Dataflow worker >>>>>>>>>> startup >>>>>>>>>> logs and make sure that correct version of Beam SDK is installed. >>>>>>>>>> >>>>>>>>>> We should understand why Python ValidatesRunner tests (which have >>>>>>>>>> passed) didn't catch this error. >>>>>>>>>> >>>>>>>>>> This can be remediated in Dataflow containers without changes to >>>>>>>>>> the release candidate. >>>>>>>>>> >>>>>>>>>> On Fri, Mar 3, 2023 at 11:22 AM Robert Bradshaw via dev < >>>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>>> >>>>>>>>>>> +1 (binding). >>>>>>>>>>> >>>>>>>>>>> I verified that the artifacts and signatures all look good, all >>>>>>>>>>> the >>>>>>>>>>> containers are pushed, and tested some pipelines with a fresh >>>>>>>>>>> install >>>>>>>>>>> from one of the Python wheels. >>>>>>>>>>> >>>>>>>>>>> On Fri, Mar 3, 2023 at 11:13 AM Danny McCormick >>>>>>>>>>> <dannymccorm...@google.com> wrote: >>>>>>>>>>> > >>>>>>>>>>> > > The released artifacts seem to be missing the last commit at >>>>>>>>>>> > > >>>>>>>>>>> https://github.com/apache/beam/commit/c528eab18b32342daed53b750fe330d30c7e5224 >>>>>>>>>>> > > . Is this essential to the release, or just useful for >>>>>>>>>>> validating it? >>>>>>>>>>> > >>>>>>>>>>> > It's strictly a test infrastructure change, it has no >>>>>>>>>>> functional impact. For context, the changes included were from >>>>>>>>>>> https://github.com/apache/beam/pull/25661 and >>>>>>>>>>> https://github.com/apache/beam/pull/25654, both were keeping >>>>>>>>>>> integration tests from running correctly. >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> > On Fri, Mar 3, 2023 at 2:09 PM Robert Bradshaw < >>>>>>>>>>> rober...@google.com> wrote: >>>>>>>>>>> >> >>>>>>>>>>> >> The released artifacts seem to be missing the last commit at >>>>>>>>>>> >> >>>>>>>>>>> https://github.com/apache/beam/commit/c528eab18b32342daed53b750fe330d30c7e5224 >>>>>>>>>>> >> . Is this essential to the release, or just useful for >>>>>>>>>>> validating it? >>>>>>>>>>> >> >>>>>>>>>>> >> On Fri, Mar 3, 2023 at 11:02 AM Danny McCormick >>>>>>>>>>> >> <dannymccorm...@google.com> wrote: >>>>>>>>>>> >> > >>>>>>>>>>> >> > Thanks for calling that out, and thanks for helping me fix >>>>>>>>>>> it! We should be all set now >>>>>>>>>>> >> > >>>>>>>>>>> >> > On Fri, Mar 3, 2023 at 1:38 PM Robert Bradshaw < >>>>>>>>>>> rober...@google.com> wrote: >>>>>>>>>>> >> >> >>>>>>>>>>> >> >> It appears your public key is not published in >>>>>>>>>>> >> >> https://dist.apache.org/repos/dist/release/beam/KEYS . >>>>>>>>>>> >> >> >>>>>>>>>>> >> >> On Fri, Mar 3, 2023 at 8:33 AM Anand Inguva via dev < >>>>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>>>> >> >> > >>>>>>>>>>> >> >> > +1 (non-binding) >>>>>>>>>>> >> >> > Tested python wordcount quick start >>>>>>>>>>> https://beam.apache.org/get-started/quickstart-py/ on Direct >>>>>>>>>>> Runner and Dataflow Runner. >>>>>>>>>>> >> >> > >>>>>>>>>>> >> >> > Thanks! >>>>>>>>>>> >> >> > >>>>>>>>>>> >> >> > On Fri, Mar 3, 2023 at 11:21 AM Bruno Volpato via dev < >>>>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> +1 (non-binding) >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> Tested with >>>>>>>>>>> https://github.com/GoogleCloudPlatform/DataflowTemplates (Java >>>>>>>>>>> SDK 11, Dataflow runner). >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> Thanks Danny! >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> On Thu, Mar 2, 2023 at 5:16 PM Danny McCormick via dev < >>>>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>>>> >> >> >>> >>>>>>>>>>> >> >> >>> Hi everyone, >>>>>>>>>>> >> >> >>> Please review and vote on release candidate #1 for the >>>>>>>>>>> version 2.46.0, as follows: [ ] +1, Approve the release [ ] -1, Do >>>>>>>>>>> not >>>>>>>>>>> approve the release (please provide specific comments) Reviewers are >>>>>>>>>>> encouraged to test their own use cases with the release candidate, >>>>>>>>>>> and vote >>>>>>>>>>> +1 if no issues are found. The complete staging area is available >>>>>>>>>>> for your >>>>>>>>>>> review, which includes: * GitHub Release notes [1], * the official >>>>>>>>>>> Apache >>>>>>>>>>> source release to be deployed to dist.apache.org [2], which is >>>>>>>>>>> signed with the key with fingerprint >>>>>>>>>>> FC383FCDE7D7E86699954EF2509872C8031C4DFB [3], * all artifacts to be >>>>>>>>>>> deployed to the Maven Central Repository [4], * source code tag >>>>>>>>>>> "v2.46.0-RC1" [5], * website pull request listing the release [6], >>>>>>>>>>> the blog >>>>>>>>>>> post [6], and publishing the API reference manual [7]. * Java >>>>>>>>>>> artifacts >>>>>>>>>>> were built with Gradle GRADLE_VERSION and OpenJDK/Oracle JDK >>>>>>>>>>> JDK_VERSION. * >>>>>>>>>>> Python artifacts are deployed along with the source release to the >>>>>>>>>>> dist.apache.org [2] and PyPI[8]. * Go artifacts and >>>>>>>>>>> documentation are available at pkg.go.dev [9] * Validation >>>>>>>>>>> sheet with a tab for 2.46.0 release to help with validation [10]. * >>>>>>>>>>> Docker >>>>>>>>>>> images published to Docker Hub [11]. >>>>>>>>>>> >> >> >>> * PR to run tests against release branch [12]. The >>>>>>>>>>> vote will be open for at least 72 hours. It is adopted by majority >>>>>>>>>>> approval, with at least 3 PMC affirmative votes. For guidelines on >>>>>>>>>>> how to >>>>>>>>>>> try the release in your projects, check out our blog post at >>>>>>>>>>> /blog/validate-beam-release/. Thanks, Danny [1] >>>>>>>>>>> https://github.com/apache/beam/milestone/9 [2] >>>>>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.46.0/ [3] >>>>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS [4] >>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1306/ >>>>>>>>>>> [5] https://github.com/apache/beam/tree/v2.46.0-RC1 [6] >>>>>>>>>>> https://github.com/apache/beam/pull/25693 [7] >>>>>>>>>>> https://github.com/apache/beam-site/pull/641 [8] >>>>>>>>>>> https://pypi.org/project/apache-beam/2.46.0rc1/ [9] >>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.46.0-RC1/go/pkg/beam >>>>>>>>>>> [10] >>>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=247587190 >>>>>>>>>>> [11] https://hub.docker.com/search?q=apache%2Fbeam&type=image >>>>>>>>>>> >> >> >>> [12] https://github.com/apache/beam/pull/25600 >>>>>>>>>>> >>>>>>>>>> >>>>>
