[PROPOSAL] Implement Kerberos support for Python and Java SDK

Hi everyone, Kerberizing workers is not an easy thing for many users, especially for Python users where some of the transforms are implemented via expansion service. I've drafted some ideas on how we could simplify this process - please read this design doc

Re: [PROPOSAL] Implement Kerberos support for Python and Java SDK

ner flow. We also may need something > similar for x-lang containers. > > Thanks, > Danny > > On Mon, Dec 16, 2024 at 7:28 AM Radek Stankiewicz via dev < > dev@beam.apache.org> wrote: > >> Hi everyone, >> >> Kerberizing workers is not an easy thing for man

[DESIGN] Beam Element Extended Metadata and future work

hi all, We’ve multiple projects in ideation, design or prototypes that share the common problem - need to extend WindowedValue with additional metadata. Those projects are: - Drain mode - https://s.apache.org/beam-drain-mode - CDC metadata - https://s.apache.org/beam-cdc-metadata (

Re: [DESIGN] Timers, Watermark Holds, Loops, Batch and Drain

hi Kenn, I would add option 3 Option 3: Use current way of using tag, add drain in additional proto object - The tag & 0b0111 indicates whether it is using FIRST, ONE_INDEX or TWO_INDICES encoding and future options if needed. - tag & 0b1 ("WITH_EXTENSION" ) indicates there i

Re: [DESIGN] Timers, Watermark Holds, Loops, Batch and Drain

Breaking change is when a portable runner (e.g. UW) running on a newer version, starts to add an extension to WindowedValue which the old SDK hasn't expected. Old SDK only expects FIRST, ONE_INDEX or TWO_INDICES, without an extension bit. Radek On Fri, Apr 25, 2025 at 4:33 PM Kenneth Knowles wro

[PR][Discussion] Open Telemetry tracing

Hi everyone, Following up on the public feature request and discussions I've decided to try to implement open telemetry tracing into Beam SDK, starting with Java SDK and two IOs - Kafka and PubSub. I've been working on this and the results can be revie