Hi,
We run Beam pipeline on Spark in the streaming mode. We subscribe to multiple
Kafka topics. Our job run fine until it is under heavy load: millions of Kafka
messages coming per seconds. The exception look like concurrency issue. Is it a
known bug in Beam or some Spark configuration we could
I havent tried yet, but looks like the connection string asks for the
project to be specified. Based on that (and cross project working for
other circumstances), I would imagine it will work, but...? Give it a try!
One tricky place might be ensuring proper permissions, in both projects
(and with
Hello
Has anyone bought this book? Can you give a simple review, good or not?
https://www.thriftbooks.com/w/apache-beam-a-complete-guide---2020-edition/26243355/item/36997139/
I want to buy a book for beam the specific topic, but this book seems
too new to have enough reviews.
Thanks.
Hi All - I am facing an issue while using *side-input*.
*What am I doing:*
>From my main program, I am calling a custom PTransform with a
PCollectionView as parameter. Inside custom PTransform, I am passing the
PCollectionView as a side-input to a DoFn.
*Issue:*
When I run the pipeline, I am expe
Hi Wesley,
I considered that one as well but was in the same boat in terms of not pulling
the trigger (lack of reviews, price point, etc.). I eventually landed on
Streaming Systems, which I highly, highly recommend if you want to learn more
about the Beam model:
- http://streamingsystems.net/
Hi Rion
Rion Williams wrote:
I considered that one as well but was in the same boat in terms of not
pulling the trigger (lack of reviews, price point, etc.). I eventually
landed on Streaming Systems, which I highly, highly recommend if you
want to learn more about the Beam model:
- http://st
