Thanks everyone, I really appreciate your contributions here. @Jason, thanks for the references I'll take a look. Quickly checking github: https://github.com/intel-analytics/analytics-zoo#distributed-tensorflow-and-keras-on-sparkbigdl Do I understand correctly I can:
- Prepare my data with Spark - Define a Tensorflow model - Train it in distributed fashion When using the Keras API, is it the real Keras with just an adapter layer or it si a completely different API that mimic Keras? @Gurav, I agree that "you should pick the right tool for the job". The purpose of this discussion is to understand/explore if we really need another stack or we can leverage on the existing infrastructure and expertise to accomplish the task. We currently have some ML jobs and Spark proved to be the perfect fit for us. We do know it enough to be confident we can deliver what is asked, it scale, it is reslient, it works. We are starting to evaluate/introduce some DL models, being able to leverage on the existing infra it would be a big plus. It is not only having to deal with a new set of machines running a different stack (ie tensorflow, mxnet, ...) it is everything around it, tuning, managing, packing applications, testing and so on. Are reasonable concerns? Best, On Sun, May 5, 2019 at 8:06 PM Gourav Sengupta <gourav.sengu...@gmail.com> wrote: > If someone is trying to actually use deep learning algorithms, their focus > should be in choosing the technology stack which gives them maximum > flexibility to try the nuances of their algorithms. > > From a personal perspective, I always prefer to use libraries which > provides the best flexibility and extensibility in terms of the science/ > mathematics of the subjects. For example try to open a book on Linear > Regression and then try to see whether all the mathematical formulations > are available in the SPARK module for regression or not. > > It is always better to choose a technology that fits into the nuances and > perfection of the science, rather than choose a technology and then try to > fit the science into it. > > Regards, > Gourav > > On Sun, May 5, 2019 at 2:23 PM Jason Dai <jason....@gmail.com> wrote: > >> You may find talks from Analytics Zoo users at >> https://analytics-zoo.github.io/master/#presentations/; in particular, >> some of recent user examples on Analytics Zoo: >> >> - Mastercard: >> >> https://software.intel.com/en-us/articles/deep-learning-with-analytic-zoo-optimizes-mastercard-recommender-ai-service >> >> - Azure: >> >> https://software.intel.com/en-us/articles/use-analytics-zoo-to-inject-ai-into-customer-service-platforms-on-microsoft-azure-part-1 >> - CERN: >> >> https://db-blog.web.cern.ch/blog/luca-canali/machine-learning-pipelines-high-energy-physics-using-apache-spark-bigdl >> - Midea/KUKA: >> >> https://software.intel.com/en-us/articles/industrial-inspection-platform-in-midea-and-kuka-using-distributed-tensorflow-on-analytics >> - Talroo: >> >> https://software.intel.com/en-us/articles/talroo-uses-analytics-zoo-and-aws-to-leverage-deep-learning-for-job-recommendation >> >> <https://software.intel.com/en-us/articles/talroo-uses-analytics-zoo-and-aws-to-leverage-deep-learning-for-job-recommendations> >> >> Thanks, >> -Jason >> >> On Sun, May 5, 2019 at 6:29 AM Riccardo Ferrari <ferra...@gmail.com> >> wrote: >> >>> Thank you for your answers! >>> >>> While it is clear each DL framework can solve the distributed model >>> training on their own (some better than others). Still I see a lot of >>> value of having Spark on the ETL/pre-processing part, thus the origin of my >>> question. >>> I am trying to avoid to mange multiple stacks/workflows and hoping to >>> unify my system. Projects like TensorflowOnSpark or Analytics-Zoo (to name >>> couple) feels like they can help, still I really appreciate your comments >>> and anyone that could add some value to this discussion. Does anyone have >>> experience with them? >>> >>> Thanks >>> >>> On Sat, May 4, 2019 at 8:01 PM Pat Ferrel <p...@occamsmachete.com> wrote: >>> >>>> @Riccardo >>>> >>>> Spark does not do the DL learning part of the pipeline (afaik) so it is >>>> limited to data ingestion and transforms (ETL). It therefore is optional >>>> and other ETL options might be better for you. >>>> >>>> Most of the technologies @Gourav mentions have their own scaling based >>>> on their own compute engines specialized for their DL implementations, so >>>> be aware that Spark scaling has nothing to do with scaling most of the DL >>>> engines, they have their own solutions. >>>> >>>> From: Gourav Sengupta <gourav.sengu...@gmail.com> >>>> <gourav.sengu...@gmail.com> >>>> Reply: Gourav Sengupta <gourav.sengu...@gmail.com> >>>> <gourav.sengu...@gmail.com> >>>> Date: May 4, 2019 at 10:24:29 AM >>>> To: Riccardo Ferrari <ferra...@gmail.com> <ferra...@gmail.com> >>>> Cc: User <user@spark.apache.org> <user@spark.apache.org> >>>> Subject: Re: Deep Learning with Spark, what is your experience? >>>> >>>> Try using MxNet and Horovod directly as well (I think that MXNet is >>>> worth a try as well): >>>> 1. >>>> https://medium.com/apache-mxnet/distributed-training-using-apache-mxnet-with-horovod-44f98bf0e7b7 >>>> 2. >>>> https://docs.nvidia.com/deeplearning/dgx/mxnet-release-notes/rel_19-01.html >>>> 3. https://aws.amazon.com/mxnet/ >>>> 4. >>>> https://aws.amazon.com/blogs/machine-learning/aws-deep-learning-amis-now-include-horovod-for-faster-multi-gpu-tensorflow-training-on-amazon-ec2-p3-instances/ >>>> >>>> >>>> Ofcourse Tensorflow is backed by Google's advertisement team as well >>>> https://aws.amazon.com/blogs/machine-learning/scalable-multi-node-training-with-tensorflow/ >>>> >>>> >>>> Regards, >>>> >>>> >>>> >>>> >>>> On Sat, May 4, 2019 at 10:59 AM Riccardo Ferrari <ferra...@gmail.com> >>>> wrote: >>>> >>>>> Hi list, >>>>> >>>>> I am trying to undestand if ti make sense to leverage on Spark as >>>>> enabling platform for Deep Learning. >>>>> >>>>> My open question to you are: >>>>> >>>>> - Do you use Apache Spark in you DL pipelines? >>>>> - How do you use Spark for DL? Is it just a stand-alone stage in >>>>> the workflow (ie data preparation script) or is it more integrated >>>>> >>>>> I see a major advantage in leveraging on Spark as a unified >>>>> entrypoint, for example you can easily abstract data sources and leverage >>>>> on existing team skills for data pre-processing and training. On the flip >>>>> side you may hit some limitations including supported versions and so on. >>>>> What is your experience? >>>>> >>>>> Thanks! >>>>> >>>>
