Yes, with_hot_key_fanout only performs a single level of fanout. I don't
think fanning out more than this has been explored, but I would imagine
that for most cases the increased IO would negate most if not all of the
benefits.
In particular, note that we already do "combiner lifting" to do as muc
We have some use-cases where we are combining over very large sets (e.g.,
computing the average of 1e5 to 1e6 elements, corresponding to hourly
weather observations over the past 50 years).
"with_hot_key_fanout" seems to be rather essential for performing these
calculations, but as far as I can te
Hi all,
I would like to update everyone on a small DNS change for the Tour Of Beam,
a new Beam interactive learning app we've been working on. We are excited
to get it live and share it with the community, and one of the remaining
steps for us is to make a DNS record for 'tour.beam.apache.org' and
Just want to clarify that Beam's concept of windowing is really an
event-time based key, and they are all processed logically simultaneously.
SQL's concept of windowing function is to sort rows and process them
linearly. They are actually totally different. From your queries it seems
you are intere
Hi Alexey,
Thank You for reference to that discussion I do actually have pretty
similar thoughts on what Beam SQL needs.
Update from my side:
Actually did find a workaround for issue with windowing function on
stream. It basically boils down to using sliding window to collect and
aggregate
It looks like some Dataflow containers didn't get published, so some jobs
using the legacy runner (runner v2 disabled) will fail. I kicked off the
container release, so that should hopefully be available later today.
Thanks,
Danny
On Thu, May 25, 2023 at 11:19 PM Ritesh Ghorse via dev
wrote:
>
This is your daily summary of Beam's current high priority issues that may need
attention.
See https://beam.apache.org/contribute/issue-priorities for the meaning and
expectations around issue priorities.
Unassigned P1 Issues:
https://github.com/apache/beam/issues/26723 [Failing Test]: Tou
