Hi Conner,
What is preventing you from using a cluster model?
I wonder if docker containers could help you here?
A quick internet search yielded Mist: https://github.com/Hydrospheredata/mist
Could be useful?
Taylor
-Original Message-
From: conner
Sent: Saturday, November 3, 2018 2:04
Hi,
What does your Spark deployment architecture looks like? Standalone? Yarn?
Mesos? Kubernetes? Those have resource managers (not middlewares) that allow to
implement scenarios as you want to achieve.
In any case you can try the FairScheduler of any of those solutions.
Best regards
> Am 03
On Sat, Nov 03, 2018 at 02:04:01AM -0700, conner wrote:
> My solution is to find a good way to divide the spark cluster resource
> into two.
What about yarn and its queue management system ?
--
nicolas
-
To unsubscribe e-mail:
