Re: Custom executor support

2015-03-11 Thread Bill Farner
The client offers no support for this currently, but the scheduler will work fine - it is ignorant of executors. So, while you would need to roll up your sleeves, you shouldn't need to move a mountain to get this working. -=Bill On Wed, Mar 11, 2015 at 9:57 AM, Soni,Mohit wrote: > I would like

installing Aurora client fails

2015-03-11 Thread Andras Kerekes
Hi All, I'm trying to install the Aurora CLI client and no success. I'm on Ubuntu 12.04 running in VirtualBox. First I tried the 0.7.0-incubating tarball version that I have downloaded from the Download section. Ran './pants binary src/main/python/apache/aurora/client/cli:aurora' command wh

Re: installing Aurora client fails

2015-03-11 Thread Bill Farner
This isn't a solution, but are you able to try to repro in a 14.04 image? This is the environment we are currently qualifying development against, so a failure there might generate more data for us. -=Bill On Wed, Mar 11, 2015 at 1:55 PM, Andras Kerekes < andras.kere...@ishisystems.com> wrote: >

Re: new features

2015-03-11 Thread Bill Farner
On Tue, Mar 10, 2015 at 6:39 AM, Haralds Ulmanis wrote: > Hello, did not find where yo open such discussion about features, but after > working with aurora here is few ideas: > 1. not pre-defined instance count. > Currently you need to specify exact number of instances on which you want > to run

Re: installing Aurora client fails

2015-03-11 Thread Hussein Elgridly
These errors look very similar, if not identical, to the ones I saw over our devops' shoulder while he found it incredibly difficult to build Aurora in a Docker container. (He gave up eventually.) You say you made it to the end of the build process - if you did indeed build the client successfully

Re: Data processing pipeline workflow management

2015-03-11 Thread Bill Farner
I'm afraid in general the use cases you describe are not things that Aurora currently intends to fulfill. Though, that's not to say that you could not do this on top of Aurora if you wanted to. Does anyone have experience with building workflows with Aurora? I do not. I could opine about how o

Re: installing Aurora client fails

2015-03-11 Thread Bill Farner
Hussein - you and/or your devops folks may benefit from this relic as a guide: https://reviews.apache.org/r/28198/ -=Bill On Wed, Mar 11, 2015 at 2:52 PM, Hussein Elgridly < huss...@broadinstitute.org> wrote: > These errors look very similar, if not identical, to the ones I saw over > our devops

Re: installing Aurora client fails

2015-03-11 Thread John Sirois
On Wed, Mar 11, 2015 at 2:55 PM, Andras Kerekes < andras.kere...@ishisystems.com> wrote: > Hi All, > > > > I’m trying to install the Aurora CLI client and no success. I’m on Ubuntu > 12.04 running in VirtualBox. > > > > First I tried the 0.7.0-incubating tarball version that I have downloaded > fr

Re: Data processing pipeline workflow management

2015-03-11 Thread Zameer Manji
Hey, This is a great question. See my comments inline below. On Tue, Mar 10, 2015 at 8:28 AM, Lars Albertsson wrote: > We are evaluating Aurora as a workflow management tool for batch > processing pipelines. We basically need a tool that regularly runs > batch processes that are connected as pr

Re: installing Aurora client fails

2015-03-11 Thread John Sirois
On Wed, Mar 11, 2015 at 4:16 PM, John Sirois wrote: > > > On Wed, Mar 11, 2015 at 2:55 PM, Andras Kerekes < > andras.kere...@ishisystems.com> wrote: > >> Hi All, >> >> >> >> I’m trying to install the Aurora CLI client and no success. I’m on Ubuntu >> 12.04 running in VirtualBox. >> >> >> >> First

Re: installing Aurora client fails

2015-03-11 Thread Bill Farner
Thanks, John! -=Bill On Wed, Mar 11, 2015 at 3:16 PM, John Sirois wrote: > On Wed, Mar 11, 2015 at 2:55 PM, Andras Kerekes < > andras.kere...@ishisystems.com> wrote: > > > Hi All, > > > > > > > > I’m trying to install the Aurora CLI client and no success. I’m on Ubuntu > > 12.04 running in Virt

RE: installing Aurora client fails

2015-03-11 Thread Andras Kerekes
Thank you, Hussein! It was there indeed, I just did not know where to look. Andras -Original Message- From: Hussein Elgridly [mailto:huss...@broadinstitute.org] Sent: Wednesday, March 11, 2015 5:52 PM To: dev@aurora.incubator.apache.org Subject: Re: installing Aurora client fails These

RE: installing Aurora client fails

2015-03-11 Thread Andras Kerekes
Hi John, Thanks for looking into it so quickly. I guess I got lucky when I tried the git cloned version, because by the time I was building that I had git installed. Andras -Original Message- From: John Sirois [mailto:john.sir...@gmail.com] Sent: Wednesday, March 11, 2015 6:25 PM To:

Re: Data processing pipeline workflow management

2015-03-11 Thread Mattmann, Chris A (3980)
Apache OODT now has a workflow plugin that connects to Mesos: http://oodt.apache.org/ Cross posting this to d...@oodt.apache.org so people like Mike Starch can chime in. ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software a