Re: Petastorm vs horovod vs tensorflowonspark vs spark_tensorflow_distributor

2021-06-07 Thread Gourav Sengupta
Hi Sean, thank you so much for your kind response :) Regards, Gourav Sengupta On Sat, Jun 5, 2021 at 8:00 PM Sean Owen wrote: > All of these tools are reasonable choices. I don't think the Spark project > itself has a view on what works best. These things do different things. For > example pet

Re: Petastorm vs horovod vs tensorflowonspark vs spark_tensorflow_distributor

2021-06-05 Thread Sean Owen
All of these tools are reasonable choices. I don't think the Spark project itself has a view on what works best. These things do different things. For example petastorm is not a training framework, but a way to feed data to a distributed DL training process on Spark. For what it's worth, Databricks

Petastorm vs horovod vs tensorflowonspark vs spark_tensorflow_distributor

2021-06-01 Thread Gourav Sengupta
Dear TD, Matei, Michael, Reynold, I hope all of you and your loved ones are staying safe and doing well. as a member of the community the direction from the SPARK mentors is getting to be a bit confusing for me and I was wondering if I can seek your help. We have to make long term decisions whic