Thanks Sean for your response. !! Want to add some more background here.
I am using Spark3.0+ version with Tensorflow 2.0+. My use case is not for the image data but for the Time-series data where I am using LSTM and transformers to forecast. I evaluated SparkFlow and spark_tensorflow_distributor libraries, and there has been no major development recently on those libraries. I faced the issue of version dependencies on those and had a hard time fixing the library compatibilities. Hence a couple of below doubts:- * Does Horovod have any dependencies? * Any other library which is suitable for my use case.? * Any example code would really be of great help to understand. Thanks, Vijayant From: Sean Owen <sro...@gmail.com> Sent: Wednesday, February 23, 2022 8:40 AM To: Vijayant Kumar <vijayant.ku...@mavenir.com.invalid> Cc: user @spark <user@spark.apache.org> Subject: [E] COMMERCIAL BULK: Re: TensorFlow on Spark Email is from a Free Mail Service (Gmail/Yahoo/Hotmail….) : Beware of Phishing Scams, Report questionable emails to s...@mavenir.com<mailto:s...@mavenir.com> Sure, Horovod is commonly used on Spark for this: https://horovod.readthedocs.io/en/stable/spark_include.html On Tue, Feb 22, 2022 at 8:51 PM Vijayant Kumar <vijayant.ku...@mavenir.com.invalid<mailto:vijayant.ku...@mavenir.com.invalid>> wrote: Hi All, Anyone using Apache spark with TensorFlow for building models. My requirement is to use TensorFlow distributed model training across the Spark executors. Please help me with some resources or some sample code. Thanks, Vijayant ________________________________ This e-mail message may contain confidential or proprietary information of Mavenir Systems, Inc. or its affiliates and is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies in your control and contact us by e-mailing to secur...@mavenir.com<mailto:secur...@mavenir.com>. This message contains the views of its author and may not necessarily reflect the views of Mavenir Systems, Inc. or its affiliates, who employ systems to monitor email messages, but make no representation that such messages are authorized, secure, uncompromised, or free from computer viruses, malware, or other defects. Thank You ________________________________ This e-mail message may contain confidential or proprietary information of Mavenir Systems, Inc. or its affiliates and is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies in your control and contact us by e-mailing to secur...@mavenir.com. This message contains the views of its author and may not necessarily reflect the views of Mavenir Systems, Inc. or its affiliates, who employ systems to monitor email messages, but make no representation that such messages are authorized, secure, uncompromised, or free from computer viruses, malware, or other defects. Thank You
