Re: Many topologies

2015-09-23 Thread Stephan Ewen
Hi Jack! Interesting use case. I think the answer depends a bit on how you want to make your setup: Variant 1: - If you have a YARN setup and allocate a new Flink AppMaster and dedicated workers, you can probably scale to as many topologies as you want (or as many as YARN can handle

Many topologies

2015-09-23 Thread Jack
Hello, I need to implement an engine for running a large number (several hundred) of different kinds of topologies. The topologies are defined ad-hoc by customers, hence combining the topologies is difficult. How does Flink perform with so many topologies? Thanks