[doc] Evaluating Third-Party Runtime Type Checking Libraries for Beam Python

Hey everyone, Hot on the heels of the type hinting overview doc and my talk on the same subject at Beam Summit, I've done a little bit of work looking into third-party libraries that could help streamline our type checking code and simplify maintaining that infrastructure. That doc is at https://s

Re: [Python] using strictest type hints

Hey Joey, I have a few concerns around creating a more variable approach to determining types. On the SDK side we'd be trying to pull hints from each route and then doing some comparison across each individual component of the hint to determine which is the most narrow. For the simplest case, wher

[doc] The Current State (and Future) of Beam Python Type Hinting

Hey everyone, Back in November of 2023 I wrote a doc focused on how Beam Python type hinting worked under the hood and discussing future-looking work (largely around how the typing module hints were being deprecated.) With all of that w

Re: MultimapState missing from Python SDK

Python 3.13 is in a bit of a limbo state at the moment, as you can iterate on the core SDK with it but GCP subpackages (anything that depends on the google-apitools package, more specifically) do not currently work. I'm blocked waiting on an update from the team that owns that package. So while the

Re: [Doc] Beam Python Yapf Version Upgrade

Hey everyone, Yapf has been upgraded to 0.43.0 in https://github.com/apache/beam/pull/34801, please update the version in your development environments accordingly. Thanks, Jack McCluskey On Thu, May 1, 2025 at 12:30 PM Jack McCluskey wrote: > Hey everyone, > > While starting work on Python 3

[Doc] Beam Python Yapf Version Upgrade

Hey everyone, While starting work on Python 3.13 support for Beam, I discovered that our current version of yapf, the package we use for formatting Python code within the project, has a dependency within the pinned version (0.29.0) that is fully removed in Python 3.13 and will need to be updated.

[Design Doc] Generic Remote Model Handlers for RunInference

Hey everyone, I've put together a design doc for a generic remote model handler base class [1], trying to abstract away some of the rougher edges of having inference calls to a remote service in the m

[RESULT] [VOTE] Release 2.63.0, release candidate #2

I'm happy to announce that we have unanimously approved this release. There are 6 approving votes, 3 of which are binding: * Danny McCormick * Jan Lukavský * Chamikara Jayalath * XQ Hu (non-binding) * Rohit Sinha (non-binding) * Yi Hu (non-binding) There are no disapproving votes. Thanks everyon

[VOTE] Release 2.63.0, release candidate #2

Hi everyone, Please review and vote on the release candidate #2 for the version 2.63.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.63.0, release candidate #1

ache/beam/pull/33921 Specificaly disable some tests > in Java which uses the bounded trie metric (outside of Lineage) to test the > metric type. Without this CP the test will continue to fail on release-2.63 > branch. > > Thank you and sorry for the CPs. > > > > On

Re: [VOTE] Release 2.63.0, release candidate #1

On Tue, Feb 11, 2025 at 8:30 AM Danny McCormick > wrote: > >> Here is the dev list RC voting thread >> >> -- Forwarded message - >> From: Jack McCluskey via dev >> Date: Tue, Feb 11, 2025 at 10:31 AM >> Subject: Re: [VOTE] Release 2.63.0,

Re: [VOTE] Release 2.63.0, release candidate #1

(non-binding). Tested it with a simple Dataflow ML pipeline: >>> https://github.com/google/dataflow-ml-starter/actions/runs/13205730673/job/36868592036 >>> >>> On Fri, Feb 7, 2025 at 11:46 AM Jack McCluskey via dev < >>> dev@beam.apache.org> wrote: >>&g

[VOTE] Release 2.63.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 2.63.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

2.63.0 Branch Cut

Hey everyone, The Beam 2.63.0 release branch has been cut! There are currently 9 open bugs on the milestone , 8 of which are flakey test tracking bugs and not release blocking. I will now start working on stabilizing the release branch and work towards

Beam 2.63.0 Release

Hey everyone, The next release (2.63.0) branch cut is scheduled for February 5, 2025, a little under 2 weeks from today, according to the release calendar [1]. I'd like to perform this release; I will cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. I apologiz

Re: [VOTE] Release 2.62.0, release candidate #1

+1 (non-binding) - downloaded python artifacts, ran direct and dataflow wordcount pipleines On Mon, Jan 13, 2025 at 10:54 AM Kenneth Knowles wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version 2.62.0, > as follows: > > [ ] +1, Approve the release > [ ] -1

Re: Sunsetting Beam Python 3.8 Support

2999 > > >>> 3.12: 241 > > >>> > > >>> 3.7: 395 > > >>> 3.6: 241 > > >>> 3.4: 156 > > >>> 2.7: 188 > > >>> > > >>> But note that any of our automation for 3.8 that pulls containers

Sunsetting Beam Python 3.8 Support

Hey everyone, With Python 3.8 reaching end-of-life in October, I've started the work of removing support in the Beam repository. The aim is to target Beam release 2.60.0 for this, since the expected release cut date is on October 2nd, 2024. The start of this effort is at https://github.com/apache/

BatchElements Overview Documentation

Hey everyone, I've taken some time to write up some user-focused context around Beam Python's BatchElements transform. There are some nuanced considerations with this transform, especially the stateful implementation, and it felt like it was worth writing up! The end goal for the content is to be

Re: [PROPOSAL] 2.8.1 point release due to potential data duplication in KafkaIO

Hey everyone, I'm assuming you mean 2.58.1? I'm +1 on a patch release to fix the problem though. Thanks, Jack McCluskey On Thu, Aug 15, 2024 at 1:48 PM Kenneth Knowles wrote: > Hi all > > I am proposing a point release because 2.8.0 has a critical data bug. The > KafkaIO unconditionally adds

[RESULT] [VOTE] Release 2.58.0, release candidate 2

Hey everyone, I'm happy to announce that we have unanimously approved this release. There are 5 approving votes, 3 of which are binding: * Jan Lukavský (binding) * Robert Burke (binding) * Valentyn Tymofieiev (binding) * XQ Hu * Yi Hu There are no disapproving votes. Thanks everyone! Jack McCl

Re: [VOTE] Release 2.58.0, release candidate #2

1, 2024 at 3:15 AM Jan Lukavský wrote: > >> +1 (binding) >> >> Tested FlinkRunner with Java SDK. >> >> Jan >> On 7/30/24 21:12, Jack McCluskey via dev wrote: >> >> Hi everyone, >> >> Please review and vote on the release candidate #2 fo

[VOTE] Release 2.58.0, release candidate #2

Hi everyone, Please review and vote on the release candidate #2 for the version 2.58.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.58.0, release candidate #1

ider framework. > > Filed bug: https://github.com/apache/beam/issues/31970 > > I am working on a fix right now. Requesting to release rc3 with either a > revert of affected commit or include my upcoming fix. > > Thanks, > - Jeff > > On Wed, Jul 17, 2024 at 5:35 PM Ja

Re: [VOTE] Release 2.58.0, release candidate #1

runner and runner v2): > > https://github.com/GoogleCloudPlatform/DataflowTemplates/tree/main/it/google-cloud-platform > > Thanks, > > Yi > > On Wed, Jul 17, 2024 at 5:35 PM Jack McCluskey via dev < > dev@beam.apache.org> wrote: > >> Hi everyone, >>

[VOTE] Release 2.58.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 2.58.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Beam 2.58.0 Release Branch Cut

Hey everyone, I just cut the 2.58.0 release branch ( https://github.com/apache/beam/tree/release-2.58.0) and will be working on getting the release branch stabilized. There are currently two release blockers on the release milestone ( https://github.com/apache/beam/milestone/22) with fixes in-fli

Beam 2.58.0 Release

Hey everyone, The next release (2.58.0) branch cut is scheduled for July 10, 2024, 2 weeks from today, according to the release calendar [1]. I'd like to perform this release; I will cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. Please help with the releas

Re: Image Embedding Support for MLTransform

It would help if I actually included the link to the new doc! https://docs.google.com/document/d/1n0ZyXcRjxqIb5f-wZcMBXdrkgjEfZHYu-zf8bpzE0dA/edit?usp=sharing On Wed, May 29, 2024 at 10:26 AM Jack McCluskey wrote: > Hey everyone, > > I've put together a very small overview doc on what steps n

Image Embedding Support for MLTransform

Hey everyone, I've put together a very small overview doc on what steps need to be taken to add support for generating image embeddings to the Python SDK's MLTransform suite. It's mostly organized as a reference for me while I get familiar with the framework and is relatively general, but it may h

Re: [RESULT] [VOTE] Release 2.53.0, release candidate #2

elsewhere). >> >> On Thu, Jan 4, 2024, 11:23 AM Jack McCluskey via dev >> wrote: >> >>> Hey everyone, >>> >>> Following up on this, I do need help from a PMC member for the PMC-only >>> finalization steps ( >>> https

Re: [RESULT] [VOTE] Release 2.53.0, release candidate #2

Hey everyone, Following up on this, I do need help from a PMC member for the PMC-only finalization steps ( https://github.com/apache/beam/blob/master/contributor-docs/release-guide.md#pmc-only-finalization ) Thanks, Jack McCluskey On Thu, Jan 4, 2024 at 10:27 AM Jack McCluskey wrote: > Hey ev

[RESULT] [VOTE] Release 2.53.0, release candidate #2

Hey everyone, I'm happy to announce that we have unanimously approved this release. There are nine approving votes, three of which are binding: * Jan Lukavský (binding) * Chamikara Jayalath (binding) * Robert Burke (binding) * XQ Hu * Danny McCormick * Bruno Volpato * Svetak Sundhar * Yi Hu * Joh

Re: [VOTE] Release 2.53.0, release candidate #2

t;>>> Jan >>>> On 12/27/23 14:13, Danny McCormick via dev wrote: >>>> >>>> +1 (non-binding) >>>> >>>> Tested with some example ML notebooks. >>>> >>>> Thanks, >>>> Danny >>>>

[VOTE] Release 2.53.0, release candidate #2

Happy holidays everyone, Please review and vote on the release candidate #2 for the version 2.53.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and v

Re: [VOTE] Release 2.53.0, release candidate #1

;> >> On Thu, Dec 21, 2023 at 4:30 PM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> +1 (non-binding) >>> >>> Tested against a few Beam ML notebooks using the direct runner and >>> Dataflow runner. >>> &g

[VOTE] Release 2.53.0, release candidate #1

Happy holidays everyone, Please review and vote on the release candidate #1 for the version 2.53.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and v

Re: Beam 2.53.0 Release Branch Cut

Hey everyone, Just wanted to give an update on the release. There's one potentially blocking issue left (https://github.com/apache/beam/issues/29818) that I'd like to get resolved, or at the very least, understood as a non-issue, before building RC1. Thanks, Jack McCluskey On Wed, Dec 13, 2023

Beam 2.53.0 Release Branch Cut

Hey everyone, I just cut the 2.53.0 release branch ( https://github.com/apache/beam/tree/release-2.53.0) and will be working on getting the release branch stabilized. There are two release blockers on the release milestone ( https://github.com/apache/beam/milestone/17) 1. A transform upgrade bug

State-Aware Element Batching in Python

Hey everyone, This is long-overdue, but I wanted to share a doc ( https://docs.google.com/document/d/1Rin_5Vm3qT1Mkb5PcHgTDrjXc3j0Admzi3fEGEHB2-4/edit?usp=sharing) that outlines a new implementation of BatchElements in the Python SDK that is state-aware, and therefore allows for batching elements

Beam 2.53.0 Release

Hey everyone, the next release (2.53.0) branch cut is scheduled for Dec 13, 2023, 2 weeks from today, according to the release calendar [1]. I'd like to perform this release; I will cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. Please help with the release

The Current State of Beam Python Type Hinting

Hey everyone, I put together a small doc explaining how Beam Python type hinting works + where the module needs to go in the future with changes to Python itself. This is over at https://s.apache.org/beam-python-type-hinting-overview and I'll be putting it into a few places for discoverability as

Re: [Discuss] Idea to increase RC voting participation

I'm +1 on helping explain what we mean by "validate the RC" since we're really just asking users to see if their existing use cases work along with our typical slate of tests. I don't know if offloading that work to our active validators is the right approach though, documentation/screen share of t

Re: [ANNOUNCE] New PMC Member: Valentyn Tymofieiev

Congrats Valentyn! On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming Valentyn > Tymofieiev as our newest PMC member. > > Valentyn has been contributing to Beam since 2017. Notable highlights > include his work on the Pyt

Re: [ANNOUNCE] New PMC Member: Robert Burke

Congrats Robert! On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming Robert Burke < > lostl...@apache.org> as our newest PMC member. > > Robert has been a part of the Beam community since 2017. He is our > resident Gopher,

Re: [ANNOUNCE] New PMC Member: Alex Van Boxel

Congrats, Alex! On Tue, Oct 3, 2023 at 2:49 PM XQ Hu via dev wrote: > Configurations, Alex! > > On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: > >> Hi all, >> >> Please join me and the rest of the Beam PMC in welcoming Alex Van Boxel < >> alexvanbo...@apache.org> as our newest PMC member

[RFC] Bootloader Buffered Logging

Hey everyone, I've written a small design doc around implementing some buffered logging for the Beam boot.go scripts that is available at https://s.apache.org/beam-buffered-logging. This should help surface errors that occur during worker set-up (like issues with dependency installation) that tend

Re: Go SDK Review Requested | README Rewrite

I've made a pass on it, also looping in Robert Burke for another set of eyes on it as well. On Sun, Jul 9, 2023 at 3:20 PM Damon Douglas wrote: > Good day, everyone, > > PR 27415 is a suggested > rewrite of the Go SDK README. Could someone review it? >

Re: [DISCUSS] Enable Github Discussions?

Also going to be -1 on this one, I'm not sure we pick anything up from adding a forum apart from adding another place that needs to be checked. On Tue, Jul 4, 2023 at 4:03 AM Jan Lukavský wrote: > -1 > > Totally agree with Byron and Alexey. > > Jan > On 7/3/23 21:18, Byron Ellis via dev wrote:

Re: [Proposal] Remove of Fix the Beam Dependency Check Report Job

ough in terms of actionability vs spam volume. > > On Tue, May 30, 2023 at 7:24 AM Jack McCluskey via dev < > dev@beam.apache.org> wrote: > >> Hi everyone, >> >> Just bumping this again now that the long weekend is behind us. If no one >> advocates for fixing the jo

Re: [RESULT] [VOTE] Release 2.48.0, release candidate #2

Completely forgot to circle back to this, these should published now On Wed, May 31, 2023 at 11:10 AM Jack McCluskey wrote: > I'll get that going shortly. > > On Wed, May 31, 2023 at 11:00 AM Ritesh Ghorse via dev < > dev@beam.apache.org> wrote: > >> Could someone from the `beammaintainers Docke

Re: [RESULT] [VOTE] Release 2.48.0, release candidate #2

I'll get that going shortly. On Wed, May 31, 2023 at 11:00 AM Ritesh Ghorse via dev wrote: > Could someone from the `beammaintainers DockerHub team` please help deploy > the docker images for release [1] > > Thanks! > > [1] > https://beam.apache.org/contribute/release-guide/#9-prepare-documents:

Client-Side Throttling in Apache Beam

Hey everyone, While working on some remote model handler code I hit a point where I needed to understand how Beam IOs interpret and action on being throttled by an external service. This turned into a few discussions and then a small write-up doc ( https://docs.google.com/document/d/1ePorJGZnLbNCm

Re: [Proposal] Remove of Fix the Beam Dependency Check Report Job

Hi everyone, Just bumping this again now that the long weekend is behind us. If no one advocates for fixing the job in the next few days I'll assume a lazy consensus and remove it. I also want to point out a typo in the subject, it should be "Remove *or* Fix." Thanks, Jack McCluskey On Thu, M

[Proposal] Remove of Fix the Beam Dependency Check Report Job

Hey everyone, The Beam Dependency Check Report email (like https://lists.apache.org/thread/tc9v1d66rx77wzvrjnkcf0jo3rxtmrhn) has not had a successful incarnation since July 21st, 2022. I've done a little bit of digging into the problem and have found that the issue lies in a query

Repository Deletion Postmortem

Hey everyone, Now that 2.47.0 is out, I've got a short write-up on how the repository was very briefly deleted back when I was cutting the release branch at https://s.apache.org/beam-repo-deletion-postmortem. This includes the rough order of operations, the recovery, and then the action items that

[RESULT] [VOTE] Release 2.47.0, release candidate #3

I'm happy to announce that we have unanimously approved this release. There are nine approving votes, four of which are binding: * Ahmet Altay * Jean-Baptiste Onofré * Chamikara Jayalath * Jan Lukavský Thank you to the following non-binding voters for helping with validation: * Anand Inguva * Aus

Re: [VOTE] Release 2.47.0, release candidate #3

ang jobs using the RC. >>>> Created https://github.com/apache/beam/issues/26576. >>>> >>>> Thanks, >>>> Cham >>>> >>>> On Fri, May 5, 2023 at 11:11 PM Austin Bennett >>>> wrote: >>>> >>>>> +

[VOTE] Release 2.47.0, release candidate #3

Hi everyone, Please review and vote on the release candidate #3 for the version 2.47.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.47.0, release candidate #2

y >> ) >> >> On Wed, May 3, 2023 at 2:01 PM Jack McCluskey via dev < >> dev@beam.apache.org> wrote: >> >>> I have just now realized I didn't replace the release version or RC >>> number from the template's first line in either vote email

Re: [PROPOSAL] Preparing for 2.48.0 Release

I'd suggest shooting for 2.48.0 so we're ahead of the end-of-support date. We're also supporting 5 different Python versions in 2.47.0, it's probably for the best to try and pare that down. On Thu, May 4, 2023 at 11:25 AM Anand Inguva via dev wrote: > Thanks Ritesh!! > > Python 3.7 support is go

Re: [Proposal] Automate Release Signing

+1 to automating release signing. As it stands now, this step requires a PMC member to add a new release manager's GPG key which can add time to getting a release started. This also results in the public key used to sign each release changing from one version to the next, as different release manag

Re: [VOTE] Release 2.47.0, release candidate #2

I have just now realized I didn't replace the release version or RC number from the template's first line in either vote email. Oops. To be clear this is 2.47.0 RC2 On Wed, May 3, 2023 at 1:58 PM Jack McCluskey wrote: > Hi everyone, > > Please review and vote on the release candidate #3 for the

[VOTE] Release 2.47.0, release candidate #2

Hi everyone, Please review and vote on the release candidate #3 for the version 1.2.3, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no i

Re: [VOTE] Release 2.47.0, release candidate #1

om/Talend/beam-samples/ > (Java SDK v8/v11/v17, Spark 3.x runner). > > --- > Alexey > > On 28 Apr 2023, at 16:06, Jack McCluskey via dev > wrote: > > There was a GCP outage that impacted pushing containers to GCR, I expected > it to impact Java containers specifically

Re: [VOTE] Release 2.47.0, release candidate #1

facts and signatures all look good, and I validated a couple of >> Python pipelines in a fresh install. >> >> Assuming all the tests (including the Dataflow ones) pass (modulo the two >> mentioned above; seems a fair justification to not block on those) I'm +1 >> (bin

Re: [VOTE] Release 2.47.0, release candidate #1

On Wed, Apr 26, 2023 at 3:23 PM Valentyn Tymofieiev > wrote: > >> Thanks, Jack! >> >> re [12]: >> >> I am seeing some test errors - have they been investigated? >> Also, did all test suites run? I think I am not seeing output of some of >> the suite

Re: [VOTE] Release 2.47.0, release candidate #1

gt; Thanks, Jack! > > re [12]: > > I am seeing some test errors - have they been investigated? > Also, did all test suites run? I think I am not seeing output of some of > the suites, like > > Run Python Dataflow V2 ValidatesRunner > > > > On Wed, Apr 26, 2023

[VOTE] Release 2.47.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #3 for the version 1.2.3, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no i

Re: [ANNOUNCE] New committer: Anand Inguva

Congratulations, Anand! On Fri, Apr 21, 2023 at 2:28 PM Valentyn Tymofieiev via dev < dev@beam.apache.org> wrote: > Congratulations! > > On Fri, Apr 21, 2023 at 8:19 PM Jan Lukavský wrote: > >> Congrats Anand! >> On 4/21/23 20:05, Robert Burke wrote: >> >> Congratulations Anand! >> >> On Fri, Ap

Re: [PROPOSAL] Preparing for 2.47.0 Release

> On Wed, Apr 12, 2023 at 8:10 PM Ahmet Altay wrote: >> >>> Jack, how is the release coming along? >>> >>> On Tue, Apr 4, 2023 at 12:23 PM Jack McCluskey via dev < >>> dev@beam.apache.org> wrote: >>> >>>> Hey everyone, >>>> &

Re: Beam Release DockerHub Group

+1 to simplifying the infra side, especially with an aim towards automating the processes we can. The more we can streamline and simplify the better. On Mon, Apr 17, 2023 at 11:18 AM Danny McCormick via dev < dev@beam.apache.org> wrote: > Hey everyone, in an effort to reduce the burden of running

Re: [PROPOSAL] Preparing for 2.47.0 Release

We're making good progress on finding and fixing bugs. Not quite to building an RC candidate yet, but so far nothing that seems to be a difficult fix. On Wed, Apr 12, 2023 at 8:10 PM Ahmet Altay wrote: > Jack, how is the release coming along? > > On Tue, Apr 4, 2023 at 12:23 PM

Re: Jenkins Flakes

Is there a technical reason we can't migrate Java code coverage over to the Codecov tool/Actions like we have with Go and Python? On Tue, Apr 11, 2023 at 11:25 AM Moritz Mack wrote: > Yes, sorry Robert for being so unspecific. With everywhere I meant Java > only, my bad! > > > > On 11.04.23, 17:

Re: Jenkins Flakes

+1 for getting Jenkins back into a happier state, getting release blockers resolved ahead of building an RC has been severely hindered by Jenkins not picking up tests or running them properly. On Tue, Apr 11, 2023 at 10:24 AM Danny McCormick via dev < dev@beam.apache.org> wrote: > *;tldr - I want

2.47.0 Branch Cut

Hey folks, The 2.47.0 release branch has been cut! Hopefully the rest of this release is extremely uneventful. Thanks, Jack McCluskey -- Jack McCluskey SWE - DataPLS PLAT/ Dataflow ML RDU jrmcclus...@google.com

Re: 2.47.0 Release Update and Brief Code Freeze

Hey folks, Thank you for your patience and understanding. The repo has been returned to a good state and the code freeze has been lifted. I'll be *actually* cutting the release branch shortly. Thanks, Jack McCluskey On Wed, Apr 5, 2023 at 4:52 PM Jack McCluskey wrote: > Hey everyone, > > Whil

2.47.0 Release Update and Brief Code Freeze

Hey everyone, While I was working on cutting the release branch this afternoon I hit a snag with the release script, and in trying to revert the halfway-completed work on that the entire Beam repo was (briefly) deleted. This has been fixed, but as a consequence of this and the permissions on the r

Re: [PROPOSAL] Preparing for 2.47.0 Release

Hey everyone, I need a PMC member's help adding my pubkey to https://dist.apache.org/repos/dist/release/beam/KEYS as well as adding PyPI user jrmccluskey to the maintainers of the Apache Beam package. These are the last steps I have to do to complete prep for the release. Thanks, Jack McCluskey

[PROPOSAL] Preparing for 2.47.0 Release

Hey all, The next (2.47.0) release branch cut is scheduled for April 5th, 2023, according to the release calendar [1]. I will be performing this release. My plan is to cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. Please help me make sure the release goes

Re: Launch Dataflow Flex Templates from Go

As far as I know, Shivam's solution is the correct one for launching a template from Go. You're just routing your calls through a Dataflow-specific API library rather than going through the API discovery service. On Wed, Feb 15, 2023 at 9:16 AM Kerry Donny-Clark wrote: > Jack added the Go templa

Re: [Go SDK] Direct Runner Replacement: Prism

Congratulations on getting the runner to a state you're happy contributing to the main repo! I'm happy to help review PRs and get sub-packages in. Anything that helps developers and users test Beam pipelines more effectively is a welcome inclusion. Thanks, Jack McCluskey P.S. I'm glad the Prism

TensorFlow Hub Integration in RunInference

Hey everyone, I've put together some thoughts on how we would allow users to load pre-trained models from TensorFlow Hub into RunInference pipelines in a small doc: https://docs.google.com/document/d/11WpWQ_x3dpgRTvkzKS47lCXA_-PGzUBAIKNJQfLpJTg/edit?usp=sharing

Re: [Python][Bikeshed] typehint vs. type-hint vs. "type hint"

I'm in agreement that how we refer to type hints in documentation should be standardized across the board. It's a good practice for both style and clarity. Seems like it wouldn't be too hard to update our docstrings either, based on a quick search of the repo. On Mon, Nov 7, 2022 at 9:00 AM Brian

Re: [ANNOUNCE] New committer: Ritesh Ghorse

Congrats Ritesh! On Thu, Nov 3, 2022 at 10:12 PM Danny McCormick via dev wrote: > Congrats Ritesh! This is definitely well deserved! > > On Thu, Nov 3, 2022 at 8:08 PM Robert Burke wrote: > >> Woohoo! Well done Ritesh! :D >> >> On Thu, Nov 3, 2022, 5:04 PM Anand Inguva via dev >> wrote: >> >>>

Re: Custom Inference Fns in RunInference

Thank you to everyone who chimed in on the doc! The discussion was very productive, and I have since updated the design doc with some more detail based on feedback and some suggestions. Any and all feedback is welcome! Thanks, Jack McCluskey On Fri, Sep 16, 2022 at 2:45 PM Jack McCluskey wrote:

Custom Inference Fns in RunInference

Hey everyone, I'm back with a brief design doc discussing ways that users could provide custom inference functions for RunInference model handlers, which is available at https://docs.google.com/document/d/1YYGsF20kminz7j9ifFdCD5WQwVl8aTeCo0cgPjbdFNU/edit?usp=sharing

Re: [RFC] State & Timers API Design for Go SDK

Great write-up on state and timers! The solution you chose feels very in-line with how the Go SDK works. Make sure the design doc makes it onto the wiki once you've addressed any feedback! On Thu, Jul 28, 2022 at 1:49 PM Kerry Donny-Clark via dev < dev@beam.apache.org> wrote: > I think this a per