Re: Apache Bean on GCP / Forcing to use py 3.11

Hello, Are you running your pipeline from the python 3.11 environment? If you are running from a python 3.11 environment and don't use a custom docker container image, DataflowRunner(Assuming Apache Beam on GCP means Apache Beam on DataflowRunner), will use Python 3.11. Thanks, Anand

Re: Dataflow not able to find a module specified using extra_package

Can you try passing `extra_packages` instead of `extra_package` when passing pipeline options as a dict? On Tue, Dec 19, 2023 at 12:26 PM Sumit Desai via user wrote: > Hi all, > I have created a Dataflow pipeline in batch mode using Apache beam Python > SDK. I am using one non-public dependency

Update on Protobuf and GCP packages for Apache Beam Python SDK

Hi, For Apache Beam Python SDK, we updated the protobuf version to 'protobuf>=4.21.1,<4.23.0' from 'protobuf>3.12.2,<4' as Protobuf had a major upgrade in May 2022 https://protobuf.dev/news/2022-05-06/. This will take effect on the Beam 2.47.0 release. A tighter bound was placed on protobuf to ma

Re: Timeline of support for Python 3.10?

Hi, Circling back on this. Python 3.10 is available with Apache Beam 2.43.0[1]. [1] https://beam.apache.org/blog/beam-2.43.0/ Thanks On Thu, Jul 21, 2022 at 5:48 PM Lina Mårtensson wrote: > Great, thanks! > > On Thu, Jul 21, 2022 at 1:07 PM Anand Inguva > wrote: > > > > Hi, > > > > Yes, we a

Benchmark tests for the Beam RunInference API

Hi, I created a doc [1] which outlines the plan for the RunInference API[2] benchmark/performance tests. I would appreciate feedback on the following, - Models used for the benchmark tests. - Metrics calc

Re: Timeline of support for Python 3.10?

Hi, Yes, we are in the middle of adding support for Python 3.10 to the Beam SDK. The ideal deadline would be to support it by the end of September. There are some blockers on type hints[1] that we are working on as of now. You can track the Python 3.10 issue here: https://github.com/apache/beam/i