See inline
— Eila www.orielesearch.com https://www.meetup.com/Deep-Learning-In-Production Sent from my iPhone > On Apr 17, 2020, at 11:32 AM, Luke Cwik <lc...@google.com> wrote: > > > When you said you checked '/usr/local/', did you check inside the docker > container or on the VM itself? Yes. Checked as part of the DoFn function that runs on the Docker > Have you tried adding an echo command before and after your script runs and > looked for them in the stackdriver logs? No. I see the name of the setup.py that I have in the logs I can add the echo command as well > It should help you locate any errors that might have happened when executing > the custom commands. Will try that. Is it correct to target to this folder. Any other folder is ‘dedicated’ for custom downloads? Thanks Eila > >> On Thu, Apr 16, 2020 at 6:30 PM OrielResearch Eila Arich-Landkof >> <e...@orielresearch.org> wrote: >> Hi all, >> >> This is a question that I have post on user-h...@apache.org. in case, the >> other one is not a valid address. posting here again. if you have already >> received it, apologies for the spam. >> >> I hope that you are all well. I would like to copy a tools library into the >> worker machines and uses the setup.py file. I have update the >> CUSTOM_COMMANDS: >> >> CUSTOM_COMMANDS = [ >> ["wget", "-O", >> "/usr/local/sratoolkit.tar.gz","http://ftp-trace.ncbi.nlm.nih.gov/sra/sdk/current/sratoolkit.current-centos_linux64.tar.gz";], >> ["tar", "-xzf", "/usr/local/sratoolkit.tar.gz","-C","/usr/local/"]] >> When I am looking for the execution files at the /user/local folder, I can >> not find the tools that I have copied to the worker. What is the right & >> easiest way to copy tools libraries into the worker machines? I was able to >> get the expected behavior of the local runner and now it's a matter of >> finding the right way to reproduce it for the dataflow runner. >> >> I am using python 3.5 with the latest apache-beam 2.20 and latest dataflow >> Thanks a lot, eilalan >> >> -- >> Eila >> >> Meetup
