Re: E0901: Namenode [localhost:8020] not allowed, not in Oozies whitelist

Hi Shay, What's the value of the oozie.service.HadoopAccessorService.nameNode.whitelist property in your oozie-site.xml? (that's the whitelist its referring to) - Robert On Sun, Oct 14, 2012 at 2:47 AM, Katz, Shay wrote: > Hi, > > I'm trying to execute the java-main from oozie(was installed w

Re: Welcoming Mona Chitnis as Oozie committer

Congrats! - Robert On Mon, Oct 15, 2012 at 5:14 PM, Alejandro Abdelnur wrote: > Congratulations Mona, well deserved > > On Mon, Oct 15, 2012 at 4:56 PM, Mohammad Islam > wrote: > > > Hi, > > On behalf of the Oozie PPMC, I am pleased to announce that Mona > > Chitnis has been elected as a comm

Re: Error on using email action

Does your oozie-site.xml have properties named "oozie.service.ActionService.executor.ext.classes" and "oozie.service.SchemaService.wf.ext.schemas"? What are their values? - Robert On Thu, Oct 25, 2012 at 4:51 PM, Prashant Kommireddi wrote: > Hi, > > Anyone have ideas on what's going on here? >

Re: Error on using email action

out email actions > > http://pkommireddi-wsl.internal.salesforce.com:11000/oozie/docs/DG_EmailActionExtension.html > Is there any other docs on it? > > -Prashant > > On Thu, Oct 25, 2012 at 5:56 PM, Robert Kanter > wrote: > > &

Re: What is the difference between java action and mapreduce action

Hi, I'm guessing that the preferred method is the MR action; its certainly easier because you don't need to write your own driver class. Either way should be executed the same on the Hadoop cluster. - Robert On Thu, Nov 1, 2012 at 9:39 AM, SingleTon wrote: > Map Reduce job can be submitted u

Re: Oozie cluster

Hi, For high availability, the only way to do this currently is in a hot/cold setup. There's some directions on Cloudera's installation instructions for Oozie (though I think it should work for "regular" Apache Oozie as well). https://ccp.cloudera.com/display/CDH4DOC/Oozie+Installation#OozieInsta

Re: Welcoming Robert Kanter as Oozie committer

Thanks everyone! - Robert On Fri, Nov 9, 2012 at 12:30 AM, Ryota Egashira wrote: > Congratulations, Robert! > > Ryota > > On 11/7/12 11:57 PM, "Mohammad Islam" wrote: > > >Hi, > >On behalf of the Oozie PPMC, I am pleased to announce that Robert Kanter &

Re: Nested Fork and Join & how to make an error silent

Hi Etienne, I'm not sure I follow exactly your notation; what do the [ and ] brackets indicate? You can have multiple joins in your workflow, so instead of having everything go to join_end, you can have a hierarchy of joins such that each fork in your workflow is in pair with a join. There is

Re: Nested Fork and Join & how to make an error silent

fy the oozie.validate.ForkJoin to false in my job.properties > file, or > when I start the job in java? > > Thanks, > > Étienne > > > On 16 November 2012 21:59, Robert Kanter wrote: > > > Hi Etienne, > > > > > > I'm not sure I follow exactly

Re: Publish schemas on the web site ?

Good idea. Though if we put this as part of the generated site, it won't be on http://oozie.apache.org/docs/ until our next release, right? Perhaps we should have a url off the main site (e.g. http://oozie.apache.org/schemas/) and just put the schemas in there for now; once we have the next relea

Re: Oozie 3.2.0-cdh4.1.2 :credentials is null for the action

Hi Uddipan, This question is really more of a CDH question, so I've Cc'd the cdh-user list. Using Oozie 3.2.0-cdh4.1.2 with Hadoop 0.20.2-cdh3u1 is not supported. You should try to use the same cdh- version of each product. Specifically, I recommend that you update your Hadoop to the latest CDH

Re: Installation and Configuration of Oozie 3.3

Hi Matt, The oozie.service.ProxyUserService.proxyuser.hadoop.hosts and oozie.service.ProxyUserService.proxyuser.hadoop.groups properties are part of Oozie's configuration and would go in oozie-site.xml. This lets you impersonate users on the Oozie side of things. See http://oozie.apache.org/docs

Re: Specifying Main Class in a MapReduce Action

Hi Dave, Its simplest to use the action and not your own main class; that said, you can also use the action to specify your main class, which can then run the mapper/reducer (that's essentially what the action does internally). - Robert On Mon, Jan 7, 2013 at 5:32 AM, Dave Cardwell wrote:

Re: Exception accessing HDFS

Hi Sarath, Please post the full Exception so we can see why exactly its throwing it. thanks - Robert On Wed, Jan 9, 2013 at 2:06 AM, Sarath < sarathchandra.jos...@algofusiontech.com> wrote: > Hi, > > I have a workflow with a Java that accesses HDFS file system to access > some files. > While e

Re: [VOTE] Release Oozie 3.3.1 (candidate 0)

I was going to mention the release-log.txt, but you beat me to it :( + I verified the md5 + I ran the tests successfully + Built and started correctly against hadoop 1.1.1 + Submitted some of the example jobs, played with the web UI and CLI a bit +1 thanks - Robert On Thu, Jan 17, 2013 at

Re: [VOTE] Release Oozie 3.3.1 (candidate 1)

+ I verified the md5 + release-log.txt no longer says "unreleased" + I ran the tests successfully + Built and started correctly against hadoop 1.1.1 + Submitted some of the example jobs, played with the web UI and CLI a bit +1 thanks - Robert On Thu, Jan 17, 2013 at 4:29 PM, Alejandro Abde

Re: Is it possible to use Oozie Java/Shell action to put a file into hdfs?

Hi Jinye, When you run that hadoop fs command, you're copying input.log from your local FS to HDFS, so it should be visible on all of your nodes. Are you sure that you're pointing the Oozie action to the right place? When you run hadoop fs -copyFromLocal /home/cloudera/input.log /user/cloudera/in

Re: Oozie Web Console & cookies: Stop responding until cleared.

I've noticed that we get a bunch of cookies named "ys-ext-comp-" where is some number. I think these are storing the width, height, etc of the "windows" that the web UI uses. I'm not sure what causes new cookies to be written, but if its saving one cookie for each window that you open an

Re: Oozie Web Console & cookies: Stop responding until cleared.

t > > > On Wed, Feb 13, 2013 at 10:28 AM, Robert Kanter >wrote: > > > I've noticed that we get a bunch of cookies named "ys-ext-comp-" > where > > is some number. I think these are storing the width, height, etc of > > the "windows&

Oozie Meetup on March 12

Hi everyone, Cloudera will be hosting an Oozie meetup on March 12 from 2:30pm to 5:00pm in our Palo Alto office. Please join us to meet fellow Oozie users and developers and have some free food. Agenda: 1) Yahoo! Oozie team to talk about their recent HCatalog integration work 2) Cloudera Hue team

Re: How to conditionally initialize parameters?

Hi Felix, If you specify the element, you can have Oozie get the output of an action; the output has to be in java properties format and there's a few other restrictions. Try searching http://oozie.apache.org/docs/3.3.1/WorkflowFunctionalSpec.html for ; it should be similar to the Java action.

Re: Hive action failing after upgrade to hive-server2

Hi Paul, Is this same issue that you were asking about in the CDH-User list? (is this the same Paul?) Unfortunately, the Hive action in Oozie isn't compatible with Hive Server 2. We are working on this and we hope (though I cannot promise) it will be done for CDH 4.3. Hi Edward, The reason that

Re: Hive action failing after upgrade to hive-server2

> original hive-server instance, but didn't seem to help. What do I need to > do to point at the old instance (which does not currently appear to be > interfering with hive-server2)? > > Thanks, > Paul > > > -Original Message- > From: Robert Kanter [mailto:rkan

Re: Hive action failing after upgrade to hive-server2

n and he moved hive-server to a new port. My > understanding now is I need to put together a custom hive-site.xml file and > fix up the share lib, is that correct? > > Thanks, > Paul > > On Mar 4, 2013, at 5:03 PM, "Robert Kanter" wrote: > > > If the original Hiv

Re: Use sqoop action to export to 3rd party database

Hi, Can you check the log in the JobTracker UI for the launcher job and paste any error messages here? thanks - Robert On Tue, Mar 12, 2013 at 11:03 AM, Parrot Silk wrote: > Hi, > > I am tying invoke oozie workflow containing a sqoop action. > > First I try the sqoop command from command

ANNOUNCE Apache Oozie 3.3.2 release

Hi All, The Apache Oozie team is pleased to announce the release of Oozie 3.3.2. Oozie 3.3.2 has many improvements and bug fixes. The details of the release and download location can be found at http://oozie.apache.org The full release notes can be found at the download location, but some highli

Re: perform copyfromlocal

Hi Panshul, 1. The Pig script has to be in HDFS. The Oozie server nor the map reduce job it launches won't have access to your local machine to pick up the script. 2. I'm guessing you want a way for your Oozie workflow to automatically upload the script from your local machine to HDFS before the

Re: perform copyfromlocal

DistCp (and thus the DistCp action itself) are meant for copying large amounts of data and files between two Hadoop clusters or within a single cluster. As far as I know, it won't accept a local filesystem path or an ftp/sftp path. - Robert On Thu, Mar 28, 2013 at 10:02 AM, Harish Krishnan < ha

Re: perform copyfromlocal

gt; > Is it possible to load a pig script stored on S3 into a workflow, if the > cluster is an EC2 cluster. running CDH4. > > Can i specify the pig script file in the normal way as I would refer to any > normal file on S3 from within hdfs? > > Thank you for the help, > > Re

Re: Oozie Sqoop Action Error: IllegalArgumentException: JobId string : is not properly formed

If an exception happens during an action's execution, it should show up in the task tracker's log for the single map task that launched the job. This would be the same as the "external id" that Oozie reports when you get info about that job. - Robert On Fri, Apr 5, 2013 at 10:27 PM, Jarek Jarce

Re: how to use Oozie launch a M/R job using a driver and specify number of reducers

Hi Frank, If you use the Java action, you should be able to do just about anything you want in there. So, if you wanted to write your own driver class, you can use the Java action to have Oozie run it, and do all configuration for your MR job from within your driver class; if I recall correctly,

Re: how to use Oozie launch a M/R job using a driver and specify number of reducers

ying to is the following and it didn't work: > > com.MyDriver > -Dmapred.reduce.tasks=${reducer_count} > > -Original Message- > From: Robert Kanter [mailto:rkan...@cloudera.com] > Sent: Tuesday, April 16, 2013 12:50 PM > To: user@oozie.apache.org

Re: running jar file in shell script

HI Savitha, You should be able to find a more specific error message in the log for the launcher job from the JT Web UI. Can you look there and paste the error here? Though my guess is that the problem is with your shell script. Long story short, the shell action is executed on an arbitrary nod

Re: Extjs link unavailable

Thanks for pointing that out Swathi. I remember looking a while ago at the sencha website and not seeing any mention of version 2.2 anywhere. I guess they finally took down the actual zip from http://extjs.com/deploy/ext-2.2.zip so its not officially available anymore. The earliest version that

Re: Configurable filesystem

Hi Panshul, I just replied this on the CDH-user list. In the future, if you want to ask multiple lists the same question, please use a single email message so that people from both lists can discuss the issue together. I've never tested it with S3, but you can try adding it to the list of > supp

Re: Extjs link unavailable

Looks like its working again: http://extjs.com/deploy/ext-2.2.zip On Wed, May 1, 2013 at 5:20 PM, Robert Kanter wrote: > Thanks for pointing that out Swathi. > > I remember looking a while ago at the sencha website and not seeing any > mention of version 2.2 anywhere. I guess

Re: Cannot start oozie-3.3.0

Hi, I typically see the "Schema 'SA' does not exist" error when I forget to create the database. Did you run the ooziedb.sh script? And the "ClassNotFoundException: org.apache.hadoop.util.ReflectionUtils" error probably means that you didn't give Oozie the hadoop jars. I suggest that you read th

[ANNOUNCE] New alternative Oozie UI

Hi everyone, A few months ago at the Oozie Meetup we demonstrated a standalone alternative Oozie UI based on Hue. We're pleased to announce that the first public release is now available! More details, including a download link and instructions, can be found here: http://gethue.tumblr.com/post/51

Re: Welcoming Rohini Palaniswamy as Oozie committer

Congrats! On Tue, May 28, 2013 at 3:10 PM, Alejandro Abdelnur wrote: > It is my pleasure to announce that Oozie PMC has invited Rohini to become > an Oozie committer and she has accepted our invitation. > > Rohini has already made several contributions to Oozie and we are sure many > more will c

Re: Welcoming Ryota Egashira as Oozie committer

Congrats! On Tue, May 28, 2013 at 4:04 PM, jun aoki wrote: > Congratulations Ryota!! > > > > On Tue, May 28, 2013 at 3:10 PM, Alejandro Abdelnur >wrote: > > > It is my pleasure to announce that Oozie PMC has invited Ryota to become > an > > Oozie committer and he has accepted our invitation. >

Re: Server Installation of Oozie

Hi Lenin, If you specify "-secure" when calling oozie-setup.sh, it will replace server.xml (and web.xml in the WAR file) with a slightly different version that will enable SSL (HTTPS). It doesn't look like you did that though, so I'm not sure why it tried to do that. In any case, if you put the

Re: Server Installation of Oozie

apache.catalina.startup.Catalina.start(Catalina.java:581) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod

Re: Server Installation of Oozie

; > And when I do bin/oozie admin -oozie http://localhost:11000/oozie -status > > it says, > > Error: IO_ERROR : java.net.ConnectException: Connection refused > * > > > Thanks, > Lenin > > > On Wed, May 29, 2013 at 10:57 PM, Robert Kanter >wrote: > >

Re: Multiple transitions to the same node

Hi Zoraida, Oozie workflows are supposed to be Directed Acyclic Graphs(DAGs), so you can't have any loops and it must be a graph. The error says that your action named "calculateGlobalMoviesUse" has multiple "ok to" transitions. Looking at you

Re: Server Installation of Oozie

talina.startup.Catalina.start(Catalina.java:595) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke

Re: Server Installation of Oozie

Mohammed, I believe that we don't support OpenJDK; if that's not clear from the documentation, we should make it more explicit. (If I'm not correct and OpenJDK should work, we should fix that) By default, CATALINA_BASE is set to CATALINA_BASE=${OOZIE_HOME}/oozie-server, and OOZIE_HOME is determi

Re: Welcome Mona Chitnis as new Oozie PMC

Congrats! On Mon, Jun 3, 2013 at 4:12 PM, Alejandro Abdelnur wrote: > Congratulations Mona, well deserved. > > > On Mon, Jun 3, 2013 at 3:57 PM, Mohammad Islam wrote: > > > Hi Oozie community, > > It is our pleasure to announce that Mona Chitnis has joined Oozie PMC. > > Mona has consistently

Re: Fw: You're invited to Hadoop Summit 2013 Oozie Meetup

Here's the link to the meetup page for anyone who wants to RSVP: http://www.meetup.com/Oozie-Contributors/events/119569602/ - Robert On Wed, Jun 12, 2013 at 12:35 PM, Mohammad Islam wrote: > FYI .. > > - Forwarded Message - > From: Oozie contributors > To: misla...@yahoo.com > Sent: W

Re: Oozie Meetup before summit: Interested in short presentation?

Hi, And we at Cloudera are planning on doing a short presentation on Oozie HA and a demo of Hue. Will take about 30 mins total. thanks - Robert On Tue, Jun 18, 2013 at 6:02 AM, Rohini Palaniswamy wrote: > Hi, >We at Y! are planning to do a presentation on the features in soon to > be re

Re: Error while using OOZIE : Failed to start database

Hi Shalish, Did you create the database? Before starting Oozie for the first time, you should run the ooziedb.sh script. - Robert On Wed, Jun 26, 2013 at 12:02 AM, Shalish VJ wrote: > Hi > > I have an oozie installation > > Oozie has been installed 2.3.0 through deb package. > > Hadoop Vers

Re: Error while using OOZIE : Failed to start database

* - Robert On Wed, Jun 26, 2013 at 10:41 PM, Shalish VJ wrote: > Hi, > > I am using oozie : 2.3.0 > > I couldnt find ooziedb.sh script in it. > > Can u please guide me. > ------ > *From:* Robert Kanter > *To:* "user@oozie.

Re: Create custom EL function

Hi Serega, I'm actually working on a tutorial blog post for the Cloudera website on how to make an EL function. It's not quite done yet, but here's the basic steps you need to do: 1) Write the function in a new Java class. There's no interface or class to inherit from, you just make a new class

Re: Create custom EL function

s, feel free > > to contribute to Oozie via creating a patch and uploading to a JIRA - > > https://issues.apache.org/jira/browse/OOZIE > > > > > > On 7/12/13 11:12 AM, "Robert Kanter" wrote: > > > > >Hi Serega, > > > > > >

Re: Create custom EL function

script. - Robert On Mon, Jul 15, 2013 at 9:43 AM, Serega Sheypak wrote: > We are using parcels. > Also you said that I have to modify oozie-site.xml to declare my function. > > > 2013/7/15 Robert Kanter > > > If you're using Cloudera Manager, you've probably inst

Re: Create custom EL function

if I use parcels? > > > 2013/7/15 Robert Kanter > > > I believe the method I described for packages should also apply for > > parcels. Yes, you still have to modify oozie-site.xml. The only > > difference in the process between a tarball installation and a CDH > >

Fwd: oozie Fs problem

Please direct future questions to the oozie mailing list instead of to me directly. How are you determining that the files are older than 5 days? If you're checking the timestamp of the file, then you'll have to use either the Shell action or the Java action to get the timestamp; you can then eit

Re: Failed to submit jobs to hadoop cluster

Hi Ying Zhu, I think you're using the wrong port for the NameNode. Unless you changed the defaults, 50070 is the Web UI and you should be using 8020 for actually using HDFS. - Robert On Thu, Jul 25, 2013 at 3:19 AM, YingZhu wrote: > Hello list, >We have a testing hadoop cluster made

Re: java action classpath precedence

Hi Jacob, For any Hadoop property, if you want it to apply to the launcher instead of the "real" job, you have to simply prepend it with "oozie.launcher.". So, in your case, you'd have "oozie.launcher.mapreduce.user.classpath.first". - Robert On Mon, Aug 12, 2013 at 9:19 AM, Williams,Jacob wr

Re: [VOTE] Release Oozie 4.0.0 (RC1)

+1 + Verified md5 + Built and setup Oozie + Ran a bunch of the Oozie examples against Hadoop 1.2 (though the libext jars were Hadoop 1.1) + Tried some misc Oozie commands and UI clicks on the examples thanks - Robert On Mon, Aug 12, 2013 at 3:02 PM, Mona Chitnis wrote: > Excuse typo in downl

Re: Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], main() threw exception, java.io.IOException: Permission denied

Hi Ravi, If you look at the TaskTracker log for the launcher job, it will have all of the details (including a stack trace) of why you got the IOException. Please paste that here. There should be a link to it from the web console when you look at the action. thanks - Robert On Fri, Aug 16, 20

Re: [VOTE] Release Oozie 4.0.0 (RC2)

+1 + Verified md5 + Built and setup Oozie (on Java 7) + Ran a bunch of the Oozie examples against Hadoop 1.2 (though the libext jars were Hadoop 1.1) + Tried some misc Oozie commands and UI clicks on the examples + Verified that wfgen isn't there thanks - Robert On Fri, Aug 23, 2013 at 10:36

Re: [VOTE] Release Oozie 4.0.0 (RC2)

Does this mean we'll need an RC3 to include OOZIE-1514? thanks - Robert On Mon, Aug 26, 2013 at 10:54 AM, Mona Chitnis wrote: > Thanks for the clarification Bowen. I was able to reproduce the issue and > opening a blocker JIRA. > > > > Mona Chitnis > Software Engineer, Hadoop Team > Yahoo! > >

Re: Oozie capture output feature understanding issue

Hi Kasa, actionData('actionName') returns an array of key-value pairs. You'd want to do actionData('actionName')['keyName'] to get the value, and you action should produce something in java properties format, i.e. "key=value". I wrote a blog post a while back about using the Shell and Java actio

Re: What does each of the service do? documentation

There should be JavaDocs for all of the Services; you can find that here: http://oozie.apache.org/docs/3.3.2/core/apidocs/index.html While the Services can be turned off, Oozie heavily relies on many of them, and won't work properly without most of them. For example, HadoopAccessorService is used

Re: [VOTE] Release Oozie 4.0.0 (RC3)

+1 + Verified md5 + Built and setup Oozie (on Java 7) + Ran a bunch of the Oozie examples against Hadoop 1.2 (though the libext jars were Hadoop 1.1) + Tried some misc Oozie commands and UI clicks on the examples thanks - Robert On Tue, Aug 27, 2013 at 1:04 PM, Mona Chitnis wrote: > Hello Oo

Re: Multiple "ok to" transitions to the same node problem

Just to double check, your decision node is in a fork-join, right? If so, Mona is correct and OOZIE-1403 will fix this. Otherwise, you shouldn't be seeing this issue at all. In the meantime, assuming its in a fork-join, you can disable the extra fork-join checking code that's causing this issue

Re: List all submitted workflows using oozie WS

Hi, You can get a list of all coordinators, their configuration, their definition, etc from various commands either via the CLI or REST interfaces. I suggest you checkout these two pages in the documentation: CLI: http://oozie.apache.org/docs/4.0.0/DG_CommandLineTool.html REST: http://oozie.apach

Re: List all submitted workflows using oozie WS

I, I want to query plain old > REST. In worst case I would use Java API. > Is composite filter is supported? Looks like I can't fitler by: > ***PSEUDO REST request goes here*** > ' > > http://uat-oozie.lol.ru:11000/oozie/v1/jobs?filter=jobType=coordinator+statu

Re: Oozie Client

Yes, the Oozie client can be installed anywhere. When you run an Oozie command, you pass it the URL of the Oozie server; as long as you can connect from that machine to the Oozie server, you can put the client anywhere. For submitting jobs that only run a single action, you can also use the "proxy

Re: Oozie PM chair change, new chair Mohammad Islam

Congrats! On Mon, Sep 16, 2013 at 10:02 AM, Mona Chitnis wrote: > Congratulations Mohammad! > > > -- > Mona Chitnis > Yahoo! > > > > From: Ryota Egashira > To: "d...@oozie.apache.org" > Cc: "oozie-...@incubator.apache.org" ; " > oozie-us...@incubator.apache.o

Re: How to pass arbitrary configurations to MR jobs started by oozie

As a general point, if you want an MR property to apply to the launcher job, you can prefix any MR property with "oozie.launcher.*". If you want an MR property to apply to the actual job, you just specify it normally. - Robert On Thu, Sep 19, 2013 at 5:42 AM, Giridhar Addepalli wrote: > Thank

Re: Oozie 4.0 and jobTracker property

Hi Alexy, That is correct, putting the RM address instead of the JobTracker will work. By "Oozie is configured for MRv2", they mean that Oozie has to have the Hadoop 2/YARN jars instead of the Hadoop 1 jars. If you build Oozie with the "hadoop-2" profile, it will use the jars from hadoop 2.2.0-S

Re: Oozie 4.0 and jobTracker property

of having > Hadoop > 2/YARN jars installed, is there something in Oozie configuration that > specify using Hadoop 2/YARN? And the Oozie 4.0 jars itself, are they > exactly the same for the MRv1 and MRv2 ? > > Thank yiou > Alexey > > > On Wed, Sep 25, 2013 at 9:32 PM,

Re: HowTo configure purge for coordinator materializations?

Hi Serega, oozie-default.xml specifies a number of oozie services to load via the "oozie.services" property; but specifying oozie.services org.apache.oozie.service.PurgeService, org.apache.oozie.service.SchedulerService you've now removed all but 2 of them. You should remove that property fro

Re: HowTo configure purge for coordinator materializations?

l... I don't see expected behaviour. > > > 2013/10/2 Robert Kanter > > > Hi Serega, > > > > oozie-default.xml specifies a number of oozie services to load via the > > "oozie.services" property; but specifying > > oozie.services > > org.apa

Re: HowTo make Oozie respect timezone?

Hi Serega, Oozie always runs everything in "oozie.processing.timezone", which defaults to UTC. The "timezone" in the coordinator is a little misleading as it doesn't actually change the timezone; only the daylight savings time rules from this timezone are used. We typically recommend users to le

Re: HowTo configure purge for coordinator materializations?

older than this value, in days, will > be > >> purged by the PurgeService. > >> > >> > >> > >> Oozie is started, cool, I forgot about oozie-default.xml. Thank you. > >> The problem is here I still see last 900+ coordinator materialization

Re: What is significance of the Oozie MR launcher?

It's also to keep the Oozie server from being bogged down or becoming unstable. For example, if you have a bunch of workflows running Pig jobs, then you'd have the Oozie server running multiple copies of the Pig client (which is a relatively "heavy" program) directly. By moving all of the user co

Re: using global params in oozie hive action

Hi Abhishek, Hive schemas < 0.3 had already been created when we added the global section, so we couldn't change them. Is there a reason you can't use Hive schema 0.4? IIRC, its almost identical to schema 0.3 except that it doesn't require a Job Tracker or Name Node, which is what you want. -

Re: Trying to run to my first oozie program but I am getting this error

Hi Namrata, Did you set oozie.service.HadoopAccessorService.supported.filesystems to something in your oozie-site.xml? If so, what value? - Robert On Fri, Oct 25, 2013 at 4:30 PM, Namrata Narula wrote: > hi, > I am first time oozie user and working on hadoop cdh4 cluster .I was try to > run

Re: using global params in oozie hive action

tracker,name-node,job-xml and > configuration), I don't see any proper place to provide param here and if > we provide them in configuration, will they be picked by hive action as > parameters to hive query. > > Thanks > Abhishek > > > On Fri, Oct 25, 2013 at 11:19

Re: Trying to run to my first oozie program but I am getting this error

13 at 5:22 PM, Namrata Narula < >>> namratannar...@gmail.com>wrote: >>> >>> > No I havent done any modification to the existing oozie-site.xml. >>> > Do I have to include property like >>> > >>> > &

Re: Oozie workflow to use "hadoop jar" to run a main() method in an MR class

Hi, (Moving to user@oozie.apache.org list) You can use the Java action ( http://oozie.apache.org/docs/4.0.0/WorkflowFunctionalSpec.html#a3.2.7_Java_Action) to run a main-class; it sounds exactly like what you’re looking for. In addition to the documentation, you may also find this blog post I wr

Re: hi，how can I complete this DAG in a workflow using oozie？

When doing forks, I always find it helpful to write it out like this or even to draw a picture: start —> fork fork —> shell1, shell2 shell1 —> fork1 fork1 —> shell3, join0 shell2 —> fork2 shell3 —> join fork2 —> shell5, join0 join0 —> shell4 shell4 —> join shell5 —> join join —> end Th

Re: Kerberos token to java actions

Hi, An action essentially consists of two classes: The ActionExecutor and the Main. The ActionExecutor runs in Oozie and handles launching the action, checking its status, etc. The Main runs in the single map-only task “launcher job” and handles running the actual action. The delegation propaga

Re: Oozie - capture output and pass it to hive script as input

Hi, There are a number of built-in EL functions, including some that deal with dates. See http://oozie.apache.org/docs/4.0.0/WorkflowFunctionalSpec.html#a4.2_Expression_Language_Functions If there isn’t an EL function that works for you, you have two options: 1) Create your own EL function. Ooz

Re: fork/join email action

Hi Tousif, There’s been a number of improvements to the fork-join validation code to make it more robust and more lenient. However, they are only included in later versions of Oozie than what you’re using. I’d recommend that you upgrade to the latest version. Alternatively, you set oozie.valida

Re: Java-Opts are completely ignored

I remember seeing something similar with java-opts where they were being set somewhere else (mapred-site?) and declared as final, so when Oozie tried to set them, it was ignored. Perhaps you’re running into the same problem? On Thu, Dec 19, 2013 at 12:23 AM, Jose Fernandez wrote: > Hi folks, >

Re: oozie HA support

Hi, The trunk has patches for Oozie HA, but its not in any release yet. Most things should be working properly but there hasn’t been extensive testing yet. If you build the documentation from trunk, it will contain setup info for Oozie HA. On Tue, Jan 7, 2014 at 10:49 PM, 习云峰 wrote: > Hello

Re: oozie HA support

We were planning to put it in 4.1.0. On Mon, Jan 20, 2014 at 9:19 PM, Roman Shaposhnik wrote: > On Mon, Jan 20, 2014 at 11:44 AM, Robert Kanter > wrote: > > Hi, > > > > The trunk has patches for Oozie HA, but its not in any release yet. Most > > things should

Re: oozie HA support

I’m not sure; I don’t think there’s really been any discussion on the next release yet. On Sat, Jan 25, 2014 at 10:43 PM, Roman Shaposhnik wrote: > On Tue, Jan 21, 2014 at 8:54 AM, Robert Kanter > wrote: > > We were planning to put it in 4.1.0. > > Great! Any thoughts on

Re: oozie java action : Why is the nameNode required?

You would put something equivalent instead of the NN. For example, when running against the local filesystem, you’d do: file: I agree that its not exactly clear. In any case, lets keep the discussion for this on OOZIE-1695 (which you created) to keep it all in one place. On Thu, Feb 13, 201

Re: Oozie HA support with derby

Hi, You can run Oozie without an HA database. A non-HA MySQL, Oracle, or Postgres database will work just fine, other than if the database goes down, your Oozie becomes unavailable (i.e. the database is a single point of failure). The reason you can't use Derby for Oozie HA is because it doesn't

Re: Oozie HA support with derby

he > database unavailable. > Can we have the oozie database on an nfs mount, so that in case of oozie > server process outage, > the secondary process which will run on another node, can access the > database using nfs mount as well? > > Thanks, > Anand > > > On Tue

[ANNOUNCE] Apache Oozie 4.0.1 released

Hello Oozie community, The Apache Oozie team is pleased to announce the release of Oozie 4.0.1. Oozie 4.0.1 includes bug fixes and minor improvements. The details of the release and download location can be found at http://oozie.apache.org (some download mirrors may not have updated yet) Thanks

Re: Failure To Start 4.0.1 Server

Hi, I just downloaded the 4.0.1 release and tried with the 1.1.1 hadooplibs jars and it worked properly for me. In Hadoop 1.1.1, AuthenticationFilter is in the haodop-core jar; can you double check that you copied it to the libext directory and ran bin/oozie-setup.sh prepare-war? Also, I notice

Re: Hadoop 2 and Oozie 4.0 - why still reference to jobTracker?

Hi, You can put the ResourceManager's address in place of the JobTracker (this actually works in Oozie 3 as well). I had created OOZIE-1508 to better document this. Another thing that we could do to make it more clear is to add a new schema versi

Re: incorporating spark jobs into oozie?

Hi, The Java action is pretty useful for running "driver" programs for MR. i.e. Java code that configures and submits a MapReduce job. I'm not super familiar with how to submit Spark jobs, but I imagine that you can write a similar driver for a Spark Job and give it to the Java action. You'd ha

Re: Switch case not working

The discussion to this is going on in the CDH user list: https://groups.google.com/a/cloudera.org/d/topic/cdh-user/svBDxDhGSBs/discussion On Fri, May 2, 2014 at 10:45 AM, Mattie wrote: > Hi, > > I am working through the Earthquake example ( > > https://blog.cloudera.com/blog/2013/03/how-to-use

Re: oozie (4.0.1) coord application fails for hadoop 2.2.0

Hi, Do you see a catalina.out file? When Oozie fails to startup, that file usually has an Exception in it with more details. Also, Oozie 4.0.1 actually builds against Hadoop 2.3.0. The directions at that website were made for Oozie 4.0.0, which was building against 2.2.0-SNAPSHOT, to edit it to

  1   2   >