Re: Non-time based windowing

2025-02-07 Thread Joey Tran
I'll try my code on a more recent release to see if it works. Thanks for pointing that fix out On Fri, Feb 7, 2025 at 3:58 PM Joey Tran wrote: > It is not different! I am using beam version 2.50 and it looks like the > changes associated with that test were in 2.56. Were custom window types > no

Re: Non-time based windowing

2025-02-07 Thread Joey Tran
It is not different! I am using beam version 2.50 and it looks like the changes associated with that test were in 2.56. Were custom window types not supported before then? On Fri, Feb 7, 2025 at 3:44 PM Robert Bradshaw wrote: > On Thu, Feb 6, 2025 at 8:39 AM Joey Tran > wrote: > >> Thanks all f

Re: Non-time based windowing

2025-02-07 Thread Robert Bradshaw via user
On Thu, Feb 6, 2025 at 8:39 AM Joey Tran wrote: > Thanks all for the conceptual sanity check. I went ahead and tried > implementing a data-based window type. I've gotten most of the way there > but it was surprisingly difficult. Not really because of any complications > due to the window not bein

Re: Number of connections to Kafka

2025-02-07 Thread Ahmet Altay via user
It should help. Adding @Yi Hu @Steven van Rossum @Sam Whittle who would be able to give a more definitive answer. On Fri, Feb 7, 2025 at 9:07 AM Utkarsh Parekh wrote: > Hi Team, > > > > I came across this PR and wanted to check if it addresses the issue of > multiple kafka connections being c

Number of connections to Kafka

2025-02-07 Thread Utkarsh Parekh
Hi Team, I came across this PR and wanted to check if it addresses the issue of multiple kafka connections being created by Dataflow jobs. https://github.com/apache/beam/pull/32928 Would appreciate any insights. Thank you, -- Utkarsh Warning: All e-mail sent t

Re: EXTERNAL Email: Re: JMSIO support

2025-02-07 Thread Sam Whittle via user
I see that there is an issue with caching of UnboundedReaders in the UnboundedSourceAsSdfWrapper which is used by the v2 dataflow runner for executing UnboundedSources (which JMS source is). This is https://github.com/apache/beam/issues/32968. This could result in more connections as each DoFn is