Hey Amit how are you doing. I can not choose 2nd option because client will not agree for password less SSH.
And for the first one, one ETL process hosted in one machin, I am not sure if I will use the mount point. On Mon, Mar 23, 2015 at 1:17 PM, Amit Mittal <[email protected]> wrote: > Hi Bhagaban, > > You have 2 questions here: > > 1. Refer to local files: You can have those local scripts and ETL jobs > available on all the nodes. May be you have a shared mount point which is > mounted on all nodes of hadoop cluster. Then put the local scripts over > there. In your HDFS script, you give the path accordingly and it should > work. > > 2. Password less SSH: You have to have the password less SSH. If all these > servers are part of Hadoop Cluster, it would be already available for root > and hadoop users. If it's not part of cluster, you need to have the key > generated and stored in all the nodes of hadoop cluster. > Hope this helps. > > Thanks > Amit > > On Mon, Mar 23, 2015 at 12:14 PM, Bhagaban Khatai < > [email protected]> > wrote: > > > I need a quick help on Oozie. > > Here is my requirement. > > > > I have a shellscript which is present in local server and inside the > shell > > script I have few jobs (basically ETL jobs which will run in that > server). > > > > Master shellscript which is present in HDFS and from there I am trying to > > run (invoke) shellscripts and some ETL jobs from local (not present in > > HDFS). > > > > Oozie workflow: user/user1/bhagaban > > > > HDFS PATH: > > > > user/user1/bhagaban > > shell1.sh > > [ > > echo "Welcome" > > sh /home/bia/abc.sh <-- invoking the shell script which will run from > > local. > > exm /run /home/bia/adm1/ETL1.dtx <- thrid party job and it should run > from > > the local server. [exm is the common to start the ETL job from local] > > ] > > > > Basically I want to run the ETL1.dtx job (third party job from local) via > > oozie, because I need to schedule accordingly. > > > > Now how should I shedule the shell script thru oozie, because the oozie > > should pick the shell script from APP-PATH (HDFS Path). but all the > > logic/external script should run from local path. > > > > I have tried SSH action as well, but I dont know how to provide password > > while calling the SSH action node. Client is not agree for password less > > SSH. > > > > Please advice me if any otherways are possible. > > > > Your help would be greatly appriciated. > > >
