is was asked a couple years ago:
https://stackoverflow.com/questions/39790820/mixed-effects-models-in-spark-or-other-technology
But I wanted to ask again, in case anyone had an answer now.
Thanks,
Josh Goldsborough
Purna,
It's a bit tangental to your original question but heads up that Amazon EKS
is in Preview right now:
https://aws.amazon.com/eks/
I don't know if it actually allows a nice interface between k8s hosted
Spark & Lamda functions (my suspicion is it won't fix your problem), but
might be somethin