Re: Chained Job Graph Apache Beam | Dataflow

2022-06-15 Thread Evan Galpin
It may also be helpful to explore CoGroupByKey as a way of joining data, though depending on the shape of the data doing so may not fit in mem: https://beam.apache.org/documentation/transforms/java/aggregation/cogroupbykey/ - Evan On Wed, Jun 15, 2022 at 3:45 PM Bruno Volpato wrote: > Hello, >

Re: Chained Job Graph Apache Beam | Dataflow

2022-06-15 Thread Bruno Volpato
Hello, I am not sure what is the context behind your join, but I just wanted to point out that Beam SQL [1] or the Join-library extension [2] may be helpful in your scenario to avoid changing semantics or the need to orchestrate your jobs outside Beam. [1] https://beam.apache.org/documentation/ds