Re: Dedicate mesos slaves to specific aurora roles

On 16 March 2015 at 06:13, Bhuvan Arumugam wrote: > Hello, > > I've a requirement to reserve N number of slaves for specific aurora > role. When a job is scheduled using this role, it should run in one of > these N slaves. Jobs scheduled using other roles, should not run in > one of these N slave

Re: Dedicate mesos slaves to specific aurora roles

The important note from that doc is: The format is $role(/.*)?. When a job is created, the scheduler requires that the $role component matches the role field in the job configuration, and will reject the job creation otherwise. > On Mar 16, 2015, at 3:39 AM, Brian Brazil wrote: > > On 16 Marc

Need help with aurora crash

Hi, I am using aurora for running 1300 instances of jenkins CIs in a mesos cluster. Aurora is running in a HA mode with a three node cluster. A few days ago aurora crashed and the end log can be seen below (Note that I have changed the actual zookeeper IP addresses with IP1 and IP2). Appreciat

Re: Speeding up Aurora client job creation

So this has now bubbled back to the top of my TODO list and I'm actively working on it. I am entirely new to Thrift so please forgive the newbie questions... I would like to talk to the Aurora scheduler directly from my (Python) application using Thrift. Since I'm on Python 3.4 I've had to use thr

Re: Speeding up Aurora client job creation

I dug into TRequestsTransport and I get it now. Sending raw bytes across a socket is not the same as doing an HTTP POST with said bytes stuffed in the body! I guess I too will be rolling my own HTTP transport... Hussein Elgridly Senior Software Engineer, DSDE The Broad Institute of MIT and Harvar

Re: Speeding up Aurora client job creation

Just to make sure I get this correctly: You say, you cannot use the existing python client because it is python 2.7 only so you want to write a new one in python 3? Regards, Stephan From: Hussein Elgridly Sent: Monday, March 16, 2015 11:44 PM To: dev@aur

Heads up - please destroy and re-create your vagrant environment

I just committed a change that alters the scheduler command line arguments in the upstart configuration. Due to the way vagrant provisioning currently works, this means the now-removed argument will be unrecognized by new scheduler builds. Please use vagrant destroy to nuke this old environment.

Build failed in Jenkins: Aurora #942

See Changes: [wfarner] Rename beta-update subcommand to update. -- [...truncated 4690 lines...] == test session starts === platform

Re: Build failed in Jenkins: Aurora #942

Known flaky test. https://issues.apache.org/jira/browse/AURORA-1162 -=Bill On Mon, Mar 16, 2015 at 7:46 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See > > Changes: > > [wfarner] Rename beta-update subcommand to update. > > -

Re: Speeding up Aurora client job creation

I'm not quite sure I understand your question, so I'll be painfully explicit instead. I don't want to use the existing Aurora client because it's slow (Pystachio + repeated HTTP connection overheads, as detailed earlier in this thread). Instead, I want to use the Thrift interface to talk to the Au

Re: Speeding up Aurora client job creation

Exploring the possibilities - can you use python 2.7? If so, you could leverage some of the private libraries within the client and lower the surface area of what you need to build. It won't be a stable programmatic API, but you might get moving faster. I assume this is what Stephan is suggestin

Re: Speeding up Aurora client job creation

As a general rule we're trying to stick to Python 3.4. I don't imagine implementing something a THTTPClient of my own will be too difficult, especially given that I have the Aurora client's TRequestsTransport [1] for reference. [1] https://github.com/apache/incubator-aurora/blob/master/src/main/py