Hi Bhavesh, I just checked with one of the devs on the Camus team. We run the Camus job with speculative execution disabled.
Aditya ________________________________________ From: Pradeep Gollakota [pradeep...@gmail.com] Sent: Monday, February 02, 2015 11:15 PM To: users@kafka.apache.org Subject: Re: Kafka ETL Camus Question Hi Bhavesh, At Lithium, we don't run Camus in our pipelines yet, though we plan to. But I just wanted to comment regarding speculative execution. We have it disabled at the cluster level and typically don't need it for most of our jobs. Especially with something like Camus, I don't see any need to run parallel copies of the same task. On Mon, Feb 2, 2015 at 10:36 PM, Bhavesh Mistry <mistry.p.bhav...@gmail.com> wrote: > Hi Jun, > > Thanks for info. I did not get answer to my question there so I thought I > try my luck here :) > > Thanks, > > Bhavesh > > On Mon, Feb 2, 2015 at 9:46 PM, Jun Rao <j...@confluent.io> wrote: > > > You can probably ask the Camus mailing list. > > > > Thanks, > > > > Jun > > > > On Thu, Jan 29, 2015 at 1:59 PM, Bhavesh Mistry < > > mistry.p.bhav...@gmail.com> > > wrote: > > > > > Hi Kafka Team or Linked-In Team, > > > > > > I would like to know if you guys run Camus ETL job with speculative > > > execution true or false. Does it make sense to set this to false ? > > Having > > > true, it creates additional load on brokers for each map task (create a > > map > > > task to pull same partition twice). Is there any advantage to this > > having > > > it on vs off ? > > > > > > mapred.map.tasks.speculative.execution > > > > > > Thanks, > > > > > > Bhavesh > > > > > >
