Re: Spark oozie job failed on launching

448.html#gbmtm>*), or just list the contents of each and every jar file loaded by the jvm trying to run MRAppMaster.main(). Regards, Andras -- Andras PIROS Software Engineer <http://www.cloudera.com/> On Fri, Nov 4, 2016 at 7:10 AM, Saurabh Malviya (samalviy) < samal...@cisco.co

Re: MiniOozie for local dryrun or other options for doing dryrun of oozie workflows?

Hi Serega, as per *Oozie documentation <http://oozie.apache.org/docs/4.1.0/DG_CommandLineTool.html#Dryrun_of_Workflow_Job>* we can see that with -dryrun option does not create nor run a job. So for the killer feature request, I think it's not possible ATM. Regards, Andras -- A

Re: Change NN and JobTracker dynamically during runtime

vide those addresses to Oozie's job.properties. That way, we separate concerns - the load balancer can choose the JT or NN node runtime, e.g. on a round robin basis. Regards, Andras -- Andras PIROS Software Engineer <http://www.cloudera.com/> On Thu, Dec 1, 2016 at 9:29 PM, mdk-swandha wro

Re: Change NN and JobTracker dynamically during runtime

load balancer setting configured runtime) is better? Thanks, Andras -- Andras PIROS Software Engineer <http://www.cloudera.com/> On Mon, Dec 5, 2016 at 7:45 PM, mdk-swandha wrote: > Hi Alex, > > The idea is to call this external service which will find the best cluster > and

Re: Exception when setup oozie HA using virtual IP

. Thanks, and regards, Andras -- Andras PIROS Software Engineer <http://www.cloudera.com/> On Tue, Dec 6, 2016 at 4:33 AM, Dongying Jiao wrote: > Hi: > Do you have the detail steps on setting up oozie HA using virtual IP? > I setup oozie HA using virtual IP, server-1 and server-

Re: Change NN and JobTracker dynamically during runtime

t can allegedly *call another script <https://www.varnish-cache.org/lists/pipermail/varnish-misc/2012-February/021690.html>* . Andras -- Andras PIROS Software Engineer <http://www.cloudera.com/> On Wed, Dec 7, 2016 at 8:28 PM, mdk-swandha wrote: > @Andreas > > Hope

Re: oozie ha

oozie. Regards, Andras -- Andras PIROS Software Engineer <http://www.cloudera.com/> On Tue, Dec 20, 2016 at 1:10 PM, WangYQ wrote: > in > http://blog.cloudera.com/blog/2014/03/inside-apache-oozie-ha/ > > > Architecture: Access > > Usually, when you use the Oozie client

Re: is there any simple demo to show how to submit a job to a oozie server with kerberos on

Hi Wang, for details on how to configure Oozie user and Hadoop authentication, please see *this * . For Oozie SSL configuration, please take a look at *this

Re: Web UI Error

Hi Suresh, can you please provide following details: - what are the exact and minimal steps to reproduce, aka what does that exactly mean "when accessing the Oozie web client"? - what is the Oozie version you're trying? Or did you build that from master? - when trying w/ Tomcat 6.0

Re: Web UI Error

this version, and all three have this > problem. I'm about to take this to AWS but wanted to check here if anyone > has encountered it. > > The Tomcat page on port 8080 shows up very fine. > > Thank you for assisting with this, > Suresh. > > > On Mon, Mar 13,

Re: Welcome new Oozie Committers - Peter Bacsko and Peter Cseh

Congratulations Peter (Geza) and Peter Bacsko! On Mon, Mar 20, 2017 at 11:37 PM, Robert Kanter wrote: > Hi everyone, > > It is my pleasure to announce that the Oozie PMC has invited > Peter Bacsko and Peter (Geza) Cseh to become Oozie committers > and they have both accepted our invitation. > >

Re: Immutable vs mutable parameters in Oozie 4.2.0

Hi Toby, thanks for your interest on using Oozie! In order I can help you further, can you please: - add *Apache GitHub * git hashes of Oozie versions for each HDP version - add a small example of two workflows, one sets a parameter, the other reads t

Re: Amazon EMR

Hi Hitesh, thanks for your interest towards Oozie! Please take a look at the *Shell Action documentation *. Regards, Andras On Thu, Apr 6, 2017 at 2:19 PM, Hitesh Goyal wrote: > Hi team, > > I am new to oozie Workflow engine.

Re: Amazon EMR

y beginning. > Where should I install oozie ? Can I install it on my web server ? > > -----Original Message- > From: Andras Piros [mailto:andras.pi...@cloudera.com] > Sent: Thursday, April 6, 2017 6:01 PM > To: user@oozie.apache.org > Subject: Re: Amazon EMR > &g

Re: is there any doc or jira to show why we need 'timezone' in coordinator.xml

Hi Wang, here u go w/ reading the *the appropriate chapter * of Coordinator Functional Specification. Regards, Andras On Sat, Apr 8, 2017 at 10:58 AM, WangYQ wrote: > is there a

Re: Oozie 4.3.0 Launcher Uber Mode + Preemption

Hi Rohit, OOZIE-2123 is that when those parameters mapreduce.job.classloader or mapreduce.user.classpath.first are set by the user, they take precedence over oozie.action.launcher. mapreduce.job.ubertask.enable one - it will be silently set to fal

Re: OYA! - no more LauncherMappers

Hi Dongying, while implementing Oozie On YARN, we tried to avoid the most backward incompatibility issues possible. However, there are some of these still remaining as part as *OOZIE-2889 *. It is mostly targeted for Oozie 5.0.0. Can you please fi

Re: Minitest failing with java.lang.NoClassDefFoundError: Could not initialize class org.apache.oozie.util.DateUtils

Hi Stanislavs, what is the OS / JDK version you're trying with? For me, mvn clean test succeeds from minitest folder: > > --- > T E S T S > --- > Java HotSpot(TM) 64-Bit Server VM warning: ig

Re: Minitest failing with java.lang.NoClassDefFoundError: Could not initialize class org.apache.oozie.util.DateUtils

gt; "unix" > > Distro is Ubuntu 16.04.2 LTS > > On Wed, Jul 19, 2017 at 2:32 PM, Andras Piros > wrote: > > > Hi Stanislavs, > > > > what is the OS / JDK version you're trying with? > > > > For me, mvn clean

Re: Welcome new Oozie Committer - Attila Sasvari

Yy! Great job, well done, Sasi! Congrats! Andras On Thu, Jul 20, 2017 at 6:01 PM, Robert Kanter wrote: > Hi everyone, > > It is my pleasure to announce that the Oozie PMC has invited > Attila Sasvari to become an Oozie committer and he has > accepted our invitation. > > Please join me in c

Re: how to set the doneflag of input event with batch number and other variable?

Hi there, yes, should support user defined variable substitution, as you can see in TestCoordSubmitXCommand#testSubmitWithDoneFlag() . Further usage of and ple

Re: Welcoming new Oozie PMC member Satish Saley

Congrats Satish! To infinity and beyond! Artem Ervits (időpont: 2017. dec. 11., H, 21:06) ezt írta: > Well deserved, congratulations Satish! > > On Dec 11, 2017 2:49 PM, "Robert Kanter" wrote: > > > Hello Oozie community, > > > > It's my pleasure to announce that Satish Saley has become the new

Re: Oozie coordinator and kafka

Hi Serega, not to my knowledge. Would be interested on your use case, though. Would start w/ *Coordinator Input Events / Datasets * . Andras On Sat, Dec 16, 2017 at 2:54 PM, Serega Sheypak wrote: > Hi, did anyone

Re: Oozie coordinator and kafka

fault coordinator functionality, but it's limited > (almost) > > > to > > > > HDFS. > > > > Kafka (any other pub/sub or queue like rabbitMQ, whatever-MQ) makes > > > > integration contract much more flexible. > > > > I could h

Re: oozie 4.3.0 can run with hadoop3.0?

Hi there, for Oozie to run w/ Hadoop 3.0.0 we need to fix *OOZIE-2973 *. This is expected to happen in Oozie 5.0.0 which drops support for Hadoop 1.x. So no, Oozie 4.3.0 doesn't support Hadoop 3.0.0. Regards. Andras On Fri, Jan 5, 2018 at 3:25

Re: Welcoming new Oozie PMC members: Attila Sasvari, Peter Cseh, and Peter Bacsko

Congrats Attila, Peter and Geza :) well deserved! Andras On Thu, Feb 1, 2018 at 12:30 AM, Artem Ervits wrote: > Well deserved, congratulations guys!! > > On Jan 31, 2018 6:27 PM, "Robert Kanter" wrote: > > > Hello Oozie community, > > > > It's my pleasure to announce that Attila Sasvari, Peter

Re: Oozie server start failed with "Invalid TimeZone: UTC" error

Hi Michael, what's your JRE / JDK version? My first thought is that you should check your JRE's timezone settings / timezone database. For *Oracle JREs *, there is an *extensive FAQ

Re: Fatal Store Error

Hello, does the RollbackException also occur when using a database other than Derby, e.g. MySQL? I'm also wondering why you use bash scripts instead of *Oozie coordinators *. Regards, Andras On Mon, Mar 19, 2018 at 6:41 PM, Da

Re: Adding --jars to oozie workflow

Hi Fawze, actually, the element of the action can contain a comma-separated list of JAR or ZIP files. So you actually already can add more than one JAR files to the Spark application. Please see documentation on *Spark action details

[ANNOUNCE] Apache Oozie 5.0.0 released

Hello Oozie community, Apache Oozie is a workflow scheduler system to manage Apache Hadoop jobs. The Apache Oozie team is pleased to announce the release of Oozie 5.0.0. Oozie 5.0.0 includes new features, bug fixes and minor improvements. The details of the release and download location can

Re: Oozie Presentation - New Features Overview and Ambari GUI

Thanks Clay, nice job! Especially good to see how Oozie can be used for cluster deployment - never would have thought of it. On Mon, Apr 30, 2018 at 7:42 PM, Mohammad Islam wrote: > Good work Clay! > On Monday, April 30, 2018, 9:15:28 AM PDT, Peter Cseh < > gezap...@cloudera.com> wrote: >

Re: Oozie for spark jobs without Hadoop

Hi, Oozie needs HDFS to store workflow, coordinator, or bundle definitions, as well as sharelib files in a safe, distributed and scalable way. Oozie needs YARN to run almost all of its actions, Spark action being no exception. At the moment it's not feasible to install Oozie without those Hadoop

Re: Oozie 5.1.0 release and plans for the year

Good idea Gp! Thanks for volunteering as the release manager for 5.1.0. I can see *a few blockers

Re: oozie install failed due to not found oozie-sharelib.tar.gz

Hi Lian Jiang, as I'm not familiar w/ HDP, can you please post exception logs? I believe HDP 3.0 is based on Oozie 4.3.1. For a general introduction how Oozie sharelib works, please see relevant parts of the Apache Oozie documentation

Re: oozie install failed due to not found oozie-sharelib.tar.gz

gt; > Again, I *sometimes *can start oozie server successfully and port 11000 > works fine. However, oozie installation is extremely unreliable due to > multiple issues (e.g. missing oozie-sharelib.tar.gz, oozie server > shutdown). > > > Any idea why oozie server shutdown? How to debug

Re: Oozie 5.1.0 release and plans for the year

. > > On Tue, Jun 19, 2018, 4:48 AM Andras Piros wrote: > > > Good idea Gp! > > > > Thanks for volunteering as the release manager for 5.1.0. I can see *a > few > > blockers > > < > > > https://issues.apache.org/jira/issues/?jql=project%20%3

Re: Oozie actions staying in PREP status

Hello, in case there are some Oozie launcher / action applications inside YARN RM WebUI but these haven't been run, chances are you need to set oozie.launcher.mapreduce.map.memory.mb to some high value in your workflow.xml file like this

[ANNOUNCE] Apache Oozie 5.1.0 released

Hello Oozie community, Apache Oozie is a workflow scheduler system to manage Apache Hadoop jobs. The Apache Oozie team is pleased to announce the release of Oozie 5.1.0. Oozie 5.1.0 includes new features, bug fixes and minor improvements. The details of the release and download location can

Re: Oozie on Kerberos

Hi Namit, for Oozie latest 5.1.0, you can try the following: 1. setting up proper Hadoop configuration , also while installing