Hi Robert,
Thanks for the advice! Checking Flink Forward talks seems like a good idea,
will do 👍
On Sat, May 22, 2021 at 4:19 AM Robert Metzger wrote:
> Hi Yaroslav,
>
> My recommendation is to go with the 2nd pattern you've described, but I
> only have limited insights into real world producti
Hi Yaroslav,
My recommendation is to go with the 2nd pattern you've described, but I
only have limited insights into real world production workloads.
Besides the parallelism configuration, I also recommend looking into slot
sharing groups, and maybe disabling operator chaining.
I'm pretty sure so
Hi Jan, thanks for sharing this!
Just wanted to confirm: this approach works because of the task slot
sharing feature in Flink, doesn't it?
On Thu, May 20, 2021 at 1:12 AM Jan Brusch
wrote:
>
> Hi Yaroslav,
>
> here's a fourth option that we usually use: We set the default
> parallelism once wh
Hi Yaroslav,
here's a fourth option that we usually use: We set the default
parallelism once when we initially deploy the app (maybe change it a few
times in the beginning). From that point on rescale by either resizing
the TaskManager-Nodes or redistributing the parallelism over more / less
Hi everyone,
I'd love to learn more about how different companies approach specifying
Flink parallelism. I'm specifically interested in real, production
workloads.
I can see a few common patterns:
- Rely on default parallelism, scale by changing parallelism for the whole
pipeline. I guess it onl
