Hi Gourav, you mean by seting a different python environment while running pyspark?
Cheers, Gorka. ________________________________________ From: Gourav Sengupta [gourav.sengu...@gmail.com] Sent: 17 April 2019 10:06 To: Gorka Bravo Martinez Cc: user@spark.apache.org Subject: Re: Boto3 library send to pyspark Hi, there is addPyFile, and then there is python environment, try to search for using python package managers like canopy and conda. Regards, Gourav On Wed, Apr 17, 2019 at 8:50 AM Gorka Bravo Martinez <gorka.bravo.marti...@cern.ch<mailto:gorka.bravo.marti...@cern.ch>> wrote: Hi all, I would like to send a boto/boto3 library while running pyspark with yarn client mode, how is it possible? I am aware sc.addFile(<path>) can add a .py file, is it the same for a library? Cheers, Gorka. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
