Limit the number of concurrent containers for an oozie workflow

2016-05-03 Thread Per Ullberg
, is there another way to throttle the number of concurrent containers for an oozie job? I guess you would have to be able to throttle both launchers and map-reduce containers? best regards /Pelle -- *Per Ullberg* Tech Lead Odin - Uppsala Klarna AB Sveavägen 46, 111 34 Stockholm Tel: +46 8 120 120

Re: Limit the number of concurrent containers for an oozie workflow

2016-05-26 Thread Per Ullberg
sequentially? > > - Robert > > On Tue, May 3, 2016 at 12:15 AM, Per Ullberg > wrote: > > > Hi, > > > > We have an oozie workflow that imports data table by table from a RDBMS > > using sqoop. One action per table. The sqoop commands use "split by >

Re: Limit the number of concurrent containers for an oozie workflow

2016-05-26 Thread Per Ullberg
The fair scheduler would solve this issue, but we need the capacity scheduler for other reason. Would it be possible to run multiple schedulers in parallel? /Pelle On Thursday, May 26, 2016, David Morel wrote: > Le 26 mai 2016 9:04 AM, "Per Ullberg" > a écrit : > > &

Re: Limit the number of concurrent containers for an oozie workflow

2016-06-01 Thread Per Ullberg
https://issues.apache.org/jira/browse/MAPREDUCE-5583 is > what you are looking for. > > On Fri, 27 May 2016 at 00:04 Per Ullberg wrote: > > > The fair scheduler would solve this issue, but we need the capacity > > scheduler for other reason. Would it be possible to run multiple > scheduler

Re: Limit the number of concurrent containers for an oozie workflow

2016-06-01 Thread Per Ullberg
yet. /Pelle On Wed, Jun 1, 2016 at 11:31 AM, Harsh J wrote: > Given each job is an independent application on YARN, there's no way to do > that outside of a Scheduler level config. > > On Wed, 1 Jun 2016 at 14:49 Per Ullberg wrote: > > > If I understand that feature cor

Re: Create table with parameterized location

2016-06-22 Thread Per Ullberg
;LOCATION' in table location specification > > I am using Hive 1.1.0 with Cloudera 5.4.8. > > Does anyone have an idea how to solve this? > > All the best, > > Yonathan > > -- *Per Ullberg* Tech Lead Odin - Uppsala Klarna AB Sveavägen 46, 111 34 Stockholm Tel: +46 8 120 120 00 Reg no: 556737-0431 klarna.com

Re: Create table with parameterized location

2016-06-23 Thread Per Ullberg
hout quotes. Whenever I put > ${hiveconf:rootpath} inside quotes, the error changes to: > > FAILED: IllegalArgumentException Can not create a Path from an empty string > > Cheers, > > Yonathan Maalo > BT Technology, Service & Operations > Mob: 07483981150 > Email: y

Re: Oozie Job + Version Control Tool

2016-10-19 Thread Per Ullberg
; > > > What is the best to manage Oozie jobs? Is there a built-in version > control > > feature? > > > > Best, > > Goun Na > > -- *Per Ullberg* Data Vault Tech Lead Odin Uppsala +46 701612693 <+46+701612693> Klarna AB (publ) Sveavägen 46, 111 34 Stockholm Tel: +46 8 120 120 00 <+46812012000> Reg no: 556737-0431 klarna.com

Re: Oozie Job + Version Control Tool

2016-10-19 Thread Per Ullberg
ing. Could you elaborate more on your > approach? > > Thanks, > Shiva > > On Wed, Oct 19, 2016 at 2:19 PM, Per Ullberg > wrote: > > > We package our oozie jobs with maven and release artifacts to nexus. We > > keep the version number as part of the coordinator name.

Re: Oozie Job + Version Control Tool

2016-10-19 Thread Per Ullberg
${wf:appPath()}/load/kafka.properties hqlDeltaTransformationPath load/hql-delta-transform-${wf:id()}.hql Sorry if I'm blabbing /Pelle On Thu, Oct 20, 2016 at 3:52 AM, goun na wrote: > Per Ullberg, a snippet of

Re: Opinion gathering Oozie job development platform

2016-10-31 Thread Per Ullberg
apability of Eclipse > with Oozie Plugin. So far, it seems its focus is basic workflow > visualization. How do you develop oozie jobs? I would like to gather all > users experience of it. > > Thanks, > Goun Na > -- *Per Ullberg* Data Vault Tech Lead Odin Uppsala +46 7

Re: Opinion gathering Oozie job development platform

2016-10-31 Thread Per Ullberg
Unfortunately, I am > not a proficient java developer. > > 2016-10-31 18:32 GMT+09:00 Per Ullberg : > > > Hi, > > > > We use intellij but with no explicit oozie support. We materialize a lot > of > > oozie jobs using mustache templates. We write junit tests u

Re: OOzie java api

2017-04-08 Thread Per Ullberg
the decision to > execute the next java action(which contains logic). > > How can I achieve this ? What classes and methods can I use ? > I tried to go through the documentation of the api but it is very short. I > am still confused.Please help. > > Regards, > Kashif > --

Re: OOzie java api

2017-04-08 Thread Per Ullberg
Kashif Hussain wrote: > Great.That's a clean way to achieve what i want. > One more thing. Are you creating this file though java action or shell > action or fs action ? > > On Sun, Apr 9, 2017 at 1:19 AM, Per Ullberg > wrote: > > > Hi, > > > > This

Re: EL Error in touchz

2017-04-10 Thread Per Ullberg
OR variable [success_file] cannot be resolved " at the > touchz command but not at fs:exists. > Why is it so ? > > I have tried without quotes,with quotes or with double quotes.But nothing > seems to work. > Please help. > > Regards, > Kashif > -- *Per Ullberg*