Re: Looking for YARN Alternative

2019-02-20 Thread Tom Davis
I'll chime in to say I am running the standalone mode successfully in Kubernetes. The ZK coordinator is very useful in this context as you can partition a topic for max *desired* parallelism without continually running that many nodes. You could also use the "operator" pattern in Kube to create a

Re: Looking for YARN Alternative

2019-02-20 Thread Jeremiah Adams
Thanks you Jagadish! This is exactly what was looking for. I've been behind samza versions due to long dev time on a product so I often miss the v.1 documentation. Jeremiah Adams Software Engineer www.helixeducation.com Blog

Re: Looking for YARN Alternative

2019-02-19 Thread Jagadish Venkatraman
The standalone mode was introduced for this exact reason for customers who don’t want to run YARN. Have you considered running Samza in stand-alone mode? In this mode, Samza is an embedded library - very similar to Kafka Streams. https://samza.apache.org/learn/documentation/latest/deployment/stan