Re: does "Deep Learning Pipelines" scale out linearly?

Hello Andy, regarding your question, this will depend a lot on the specific task:  - for tasks that are "easy" to distribute such as inference (scoring), hyper-parameter tuning or cross-validation, these tasks will take full advantage of the cluster and the performance should improve more or less l

[ann] Release of TensorFrames 0.2.8

Dataframes) lets you manipulate Spark's DataFrames with TensorFlow programs. Spark package: https://spark-packages.org/package/databricks/tensorframes Release notes: https://github.com/databricks/tensorframes/releases/tag/v0.2.8 Best regards Tim Hunter [1] https://databricks

GraphFrames 0.2.0 released

Hello all, I have released version 0.2.0 of the GraphFrames package. Apart from a few bug fixes, it is the first release published for Spark 2.0 and both scala 2.10 and 2.11. Please let us know if you have any comment or questions. It is available as a Spark package: https://spark-packages.org/pac

Request for comments: Tensorframes, an integration library between TensorFlow and Spark DataFrames

Tim Hunter

Introducing spark-sklearn, a scikit-learn integration package for Spark

Hello community, I would like to introduce a new Spark package that should be useful for python users who depend on scikit-learn. Among other tools: - train and evaluate multiple scikit-learn models in parallel. - convert Spark's Dataframes seamlessly into numpy arrays - (experimental) distribu