Re: MiniOozie for local dryrun or other options for doing dryrun of oozie workflows?

Hi Serega, as per *Oozie documentation * we can see that with -dryrun option does not create nor run a job. So for the killer feature request, I think it's not possible ATM. Regards, Andras -- Andras PIROS Softw

Re: Change NN and JobTracker dynamically during runtime

Hi Dipesh, seems like a bad idea to programmatically change job-tracker or name-node properties - it's just not the task of Oozie to determine what are the exact JT or NN instances Oozie should use. Instead, I'd rather setup a load balancer for JT and another one for NN, and provide those address

Re: Change NN and JobTracker dynamically during runtime

Hi Alex, The idea is to call this external service which will find the best cluster and inform the caller. So today this caller is Oozie, tomorrow it will be Zeppelin or any other application. How can I provide multiple JT and NN addresses in job.properties? You mean during job/workflow creation?

Re: Change NN and JobTracker dynamically during runtime

Hi Dipesh, during workflow / job submission you can define variables inside job.properties coming e.g. from env vars that are used in workflow.xml. So much for the flexibility. Can you tell me a use case where runtime routing to different JT / NN instances via Oozie (and not e.g. coming from a lo

Re: Change NN and JobTracker dynamically during runtime

You mean I have to set env variables for each job/workflow execution and then it will be picked up by Oozie. And I should set them in my service (the service which is finding the best cluster?). For example let say I have 3 cluster: - When a job is sent via Oozie/Hue/Zepellin/Livy etc. - they are

How to run and manage a jobs like storm , flink , python that don't have inbuilt action plugin in Oozie

Hi team As per my knowledge Oozie is supporting several types of Hadoop jobs such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop , spark and Distcp , as well as system specific jobs such as Java programs and shell scripts . but i want to run and manage a jobs like storm , flink , py

Question about spark-bagel in oozie 4.3.0

Hi: I noticed oozie 4.3.0 add spark-bagel lib to spark sharelib compared to oozie 4.2.0. I found this module is deprecated and superseded by GraphX in spark official site, why do we add this deprecated compenent since GraphX is already in sharelib? And I want to use spark 2.0 for oozie 4.3.0, as t

Exception when setup oozie HA using virtual IP

Hi: Do you have the detail steps on setting up oozie HA using virtual IP? I setup oozie HA using virtual IP, server-1 and server-2(active-active), when we take down server-1 any oozie job submitted fails with below stacktrace. If both are up , there is no issue. ERROR RecoveryService$RecoveryRunnab

Re: MiniOozie for local dryrun or other options for doing dryrun of oozie workflows?

Yeah, I see it. I found the way to test wokflow locally but it's suuuper complex. I have to start local MR, Local HDFS and Local OOzie things. Then I do mock on the fly xml actions with my test actions and run workflow. It's super complex and fragile unfortunately... I'll try to reach dev group. I