Re: Looking for YARN Alternative

I'll chime in to say I am running the standalone mode successfully in Kubernetes. The ZK coordinator is very useful in this context as you can partition a topic for max *desired* parallelism without continually running that many nodes. You could also use the "operator" pattern in Kube to create a

Re: Looking for YARN Alternative

Thanks you Jagadish! This is exactly what was looking for. I've been behind samza versions due to long dev time on a product so I often miss the v.1 documentation. Jeremiah Adams Software Engineer www.helixeducation.com Blog

Re: Looking for YARN Alternative

The standalone mode was introduced for this exact reason for customers who don’t want to run YARN. Have you considered running Samza in stand-alone mode? In this mode, Samza is an embedded library - very similar to Kafka Streams. https://samza.apache.org/learn/documentation/latest/deployment/stan

Looking for YARN Alternative

We are finding YARN and AWS Ec2 to be too costly for us. We are having to scale the cluster to support more jobs and have plans to write more jobs. We are scaling because cluster doesn’t have enough VCores to support all the Containers, not enough RAM for jobs, etc. Has anyone had luck running