Beam High Priority Issue Report (42)

2023-01-06 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/24870 [Bug]: Encoding iss

Hacking a Rust SDK

2023-01-06 Thread Robert Bradshaw via dev
Hi all, Welcome to 2023! As last year, the Google Dataflow team is kicking the first week off with a hackathon, and one of the projects proposed this year was to throw together a rust SDK. If you're interested, you can follow the progress at https://github.com/kennknowles/beam/tree/rust/sdks/rust

Re: BigTable reader for Python?

2023-01-06 Thread Lina MÃ¥rtensson via dev
I am *so close* it seems. ;) I followed Luke's advice and am reading the proto com.google.bigtable.v2.Row, then use a transform to convert that to bytes in order to be able to send it across to Python. (I assume that's what I should be doing with the proto?) Once on the Python side, when running o

Re: Hacking a Rust SDK

2023-01-06 Thread Yichi Zhang via dev
I'm very interested, I admit that I know nothing about Rust but always had the desire to learn, are there github issues tracking the work? On Fri, Jan 6, 2023 at 11:09 AM Robert Bradshaw via dev wrote: > Hi all, > > Welcome to 2023! As last year, the Google Dataflow team is kicking the > first w

Re: Hacking a Rust SDK

2023-01-06 Thread Robert Bradshaw via dev
Just TODOs in the code for now. (The codebase is small/fresh enough to grep for them.) On Fri, Jan 6, 2023 at 11:23 AM Yichi Zhang wrote: > > I'm very interested, I admit that I know nothing about Rust but always had > the desire to learn, are there github issues tracking the work? > > On Fri, J

Re: Beam Website Feedback

2023-01-06 Thread Alex Kosolapov
Thank you, Ahmet! Happy to help! Both changes [1] and [2] have been reviewed and merged by Alexey Romanenko. We wanted to thank Alexey Romanenko, David Huntsperger, Pablo Estrada, Alya Boiko for reviewing and helping to contribute 52 enhancements, fixes and case study related additions for the

Re: Beam Website Feedback

2023-01-06 Thread Ahmet Altay via dev
Thank you: Alexey Romanenko, David Huntsperger, Pablo Estrada, Alya Boiko! :) On Fri, Jan 6, 2023 at 12:54 PM Alex Kosolapov wrote: > Thank you, Ahmet! Happy to help! Both changes [1] and [2] have been > reviewed and merged by Alexey Romanenko. > > > > We wanted to thank Alexey Romanenko, David

Re: BigTable reader for Python?

2023-01-06 Thread Luke Cwik via dev
The proto (java) -> bytes -> proto (python) sounds good. Have you tried moving your DoFn outside of your main module into a new module as per [1]. Other suggestions are to do the import in the function. Can you do the import once in the setup()[2] function? Have you considered using the cloud prof