Hi Yong
Have you also tried s3a in place of s3?
-
Suresh.
> On Mar 25, 2019, at 2:03 PM, Peter Cseh <gezap...@cloudera.com.invalid> wrote:
>
> Hey Yong,
>
> Thanks for reporting this issue!
> If I see correctly, your Oozie is set up to talk to a HDFS instance and to
> S3 as well. This is not a scenario I'm too familiar with.
> Could you give us some easy-to-follow steps to reproduce this?
> Thanks
> gp
>
>> On Thu, Mar 21, 2019 at 11:13 PM Daniel Zhang <java8...@hotmail.com> wrote:
>>
>> Hi, oozier:
>>
>> Since AWS EMR 5.15.0, it releases with Oozie 5.0.0, upgrades from oozie
>> 4.3.
>>
>> We found out one nice feature was broken for us on Oozie 5.0.0,
>> unfortunately.
>>
>> On Oozie 4.3, we put our oozie applications in one S3 bucket, as our
>> release repository, and in the oozie application properties file, we just
>> use as following:
>>
>> appBaseDir=${s3.app.bucket}/oozieJobs/${appName}
>>
>> And oozie 4.3 runtime will load all the application code from the S3, and
>> still use the oozie sharelib from the HDFS for us, and whole application
>> workflow works perfectly.
>>
>> After EMR 5.15.0, it upgrades to Oozie 5.0.0, and we cannot use S3 as our
>> application repository anymore. The same application will WORK fine if the
>> application is stored in HDFS. But if stored in S3, we got the following
>> error message:
>>
>> Caused by: org.apache.oozie.workflow.WorkflowException: E0712: Could not
>> create lib paths list for application
>> [s3://bucket-name/oozieJobs/ourAppName/workflow/workflow.xml], Wrong FS:
>> hdfs://ip-172-31-72-175.ec2.internal:8020/user/oozie/share/lib, expected:
>> s3://bucket-name
>> at
>> org.apache.oozie.service.WorkflowAppService.createProtoActionConf(WorkflowAppService.java:258)
>> at org.apache.oozie.command.wf
>> .SubmitXCommand.execute(SubmitXCommand.java:168)
>> ... 36 more
>> Caused by: java.lang.IllegalArgumentException: Wrong FS:
>> hdfs://ip-172-31-72-175.ec2.internal:8020/user/oozie/share/lib, expected:
>> s3://bucket-name
>> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:669)
>> at
>> org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:487)
>> at
>> com.amazon.ws.emr.hadoop.fs.staging.DefaultStagingMechanism.isStagingDirectoryPath(DefaultStagingMechanism.java:38)
>> at
>> com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem.getFileStatus(S3NativeFileSystem.java:740)
>> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1440)
>> at
>> com.amazon.ws.emr.hadoop.fs.EmrFileSystem.exists(EmrFileSystem.java:347)
>> at
>> org.apache.oozie.service.WorkflowAppService.getLibFiles(WorkflowAppService.java:301)
>> at
>> org.apache.oozie.service.WorkflowAppService.createProtoActionConf(WorkflowAppService.java:202)
>> ... 37 more
>>
>> It looks like if we config the APP path as in S3 by
>> appBaseDir=${s3.app.bucket}/oozieJobs/${appName}, Oozie 5.0 will complain
>> that it cannot load the sharelib any more from the HDFS URI, even though
>> the all the share lib are indeed stored in the HFDS correct location as
>> specified in the error message.
>>
>> With this error message, I found out the following commit in the Oozie 5.0
>>
>> https://github.com/apache/oozie/commit/5998c18fde1da769e91e3ef1bcca484723730c76#diff-d4e9af2c1e2ddeae544be6182b948109
>>
>> Since the error comes from the FileSystem in
>> core/src/main/java/org/apache/oozie/service/WorkflowAppService.java<
>> https://github.com/apache/oozie/commit/5998c18fde1da769e91e3ef1bcca484723730c76#diff-d4e9af2c1e2ddeae544be6182b948109>,
>> so I think MAYBE above commit causing it?
>> [https://avatars3.githubusercontent.com/u/2914398?s=200&v=4]<
>> https://github.com/apache/oozie/commit/5998c18fde1da769e91e3ef1bcca484723730c76#diff-d4e9af2c1e2ddeae544be6182b948109
>>>
>>
>> OOZIE-2944 Shell action example does not work with Oozie on Yarn on h… ·
>> apache/oozie@5998c18 - GitHub<
>> https://github.com/apache/oozie/commit/5998c18fde1da769e91e3ef1bcca484723730c76#diff-d4e9af2c1e2ddeae544be6182b948109
>>>
>> Mirror of Apache Oozie. Contribute to apache/oozie development by creating
>> an account on GitHub.
>> github.com
>>
>>
>> In 5.0.0, on line 202, it is using the "fs" which comes from line 177 with
>> a "conf" coming from line 169 like following:
>> https://github.com/apache/oozie/blob/branch-5.0/core/src/main/java/org/apache/oozie/service/WorkflowAppService.java#L166
>>
>> URI uri = new URI(jobConf.get(OozieClient.APP_PATH));
>>
>> Configuration conf =
>> has.createConfiguration(uri.getAuthority());
>>
>>
>> But in 4.3.0 at
>> https://github.com/apache/oozie/blob/branch-4.3/core/src/main/java/org/apache/oozie/service/WorkflowAppService.java#L167
>>
>>
>> URI uri = new URI(jobConf.get(OozieClient.APP_PATH));
>>
>> Configuration conf = has.createJobConf(uri.getAuthority());
>>
>>
>> I am NOT 100% sure, but the above code indeed returns the FileSystem
>> eventually complains "WRONG FS" in my case, and the above commit changes
>> the "jobConf" from the createJobConf to createConfiguration.
>>
>> So my question here, do you think that it is the above change causing my
>> issue? If so, I believe there is a reason for the above commit, but do I
>> have a solution also for my use case?
>>
>> Thanks
>>
>> Yong
>>
>>
>
> --
> *Peter Cseh *| Software Engineer
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
