Hi, Thanks for your patience. I have been consumed with spark-swift, but I can start to address these questions now :)
On (1) (a) below, I will try to reproduce and look at how we can better support classpath in EDP. I'll let you know what I find. We may need to add some configuration options for EDP or change how it works. On (1) (b) below, in the edp-move-examples.rst spec for Juno we described a directory structure that could be used for separating hadoop1 vs hadoop2 specific directories. Maybe we can do something similar based on plugins For instance, if we have some hbase examples, we can make subdirectories for each plugin. Common parts can be shared, plugin-specific files can be stored in the subdirectories. (and perhaps the "hadoop2" example already there should just be a subdirectory under "edp-java") Best, Trevor -- Hi McKay Thx for your support I will talk details of these items as below: (1) EDP job in Java action The background is that we want write integration test case for newly added services like HBase, zookeeper just like the way the edp-examples does( sample code under sahara/etc/edp-examples/). So I thought I can wrote an example via edp job by Java action to test HBase Service, then I wrote the HBaseTest.java and packaged as a jar file, and run this jar manually with the command "java -cp `hbase classpath` HBaseTest.jar HBaseTest", it works well in the vm(provisioned by sahara with cdh plugin). “/usr/lib/jvm/java-7-oracle-cloudera/bin/java -cp "HBaseTest.jar:`hbase classpath`" HBaseTest” So I want run this job via horizon in sahara job execution page, but found no place to pass the `hbase classpath` parameter.(I have tried java_opt and configuration and args, all failed). When I pass the “-cp `hbase classpath`” to java_opts in horizon job execution page. Oozie raise this error as below “2015-01-15 16:43:26,074 WARN org.apache.oozie.action.hadoop.JavaActionExecutor: SERVER[hbase-master-copy-copy-001.novalocal] USER[hdfs] GROUP[-] TOKEN[] APP[job-wf] JOB[0000045-150105050354389-oozie-oozi-W] ACTION[0000045-150105050354389-oozie-oozi-W@job-node] LauncherMapper died, check Hadoop LOG for job [hbase-master-copy-copy-001.novalocal:8032:job_1420434100219_0054] 2015-01-15 16:43:26,172 INFO org.apache.oozie.command.wf.ActionEndXCommand: SERVER[hbase-master-copy-copy-001.novalocal] USER[hdfs] GROUP[-] TOKEN[] APP[job-wf] JOB[0000045-150105050354389-oozie-oozi-W] ACTION[0000045-150105050354389-oozie-oozi-W@job-node] ERROR is considered as FAILED for SLA” So I stuck with this issue, I can’t write the integration test in sahara ( could not pass the classpath parameter), I have check oozie official site . https://cwiki.apache.org/confluence/display/OOZIE/Java+Cookbook found no help info. So about the EDP job in java, I have two problems right now: a) How to pass classpath to java action as I mention before. So this also reminds me that we can allow user to modify or upload this own workflow.xml, then we can provide more options for user. b) I concern that it’s hard to have a common edp-example for HBase for all plugin(cdh dhp), because the example code depends on third party jars(for example hbase-client.jar…) and different platform(CDH HDP) they may have different version hbase-client.jar, for example, cdh use hbase-client-0.98.6-cdh5.2.1.jar. attached is a zip file which contains HBaseTest.jar and the source code.
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
