Hi all, This is a question that I have post on user-h...@apache.org. in case, the other one is not a valid address. posting here again. if you have already received it, apologies for the spam.
I hope that you are all well. I would like to copy a tools library into the worker machines and uses the setup.py file. I have update the CUSTOM_COMMANDS: CUSTOM_COMMANDS = [ ["wget", "-O", "/usr/local/sratoolkit.tar.gz","http://ftp-trace.ncbi.nlm.nih.gov/sra/sdk/current/sratoolkit.current-centos_linux64.tar.gz";], ["tar", "-xzf", "/usr/local/sratoolkit.tar.gz","-C","/usr/local/"]] When I am looking for the execution files at the /user/local folder, I can not find the tools that I have copied to the worker. What is the right & easiest way to copy tools libraries into the worker machines? I was able to get the expected behavior of the local runner and now it's a matter of finding the right way to reproduce it for the dataflow runner. I am using python 3.5 with the latest apache-beam 2.20 and latest dataflow Thanks a lot, eilalan -- Eila <http://www.orielresearch.com> Meetup <https://www.meetup.com/Deep-Learning-In-Production/>
