Re: mllib + SQL

What are you interested in accomplishing? The spark.ml package has provided a machine learning API based on DataFrames for quite some time. If you are interested in mixing query processing and machine learning, this is certainly the best place to start. See here: https://spark.apache.org/docs/l

Re: SPIP: Spark on Kubernetes

+1 (non-binding) On Tue, Aug 15, 2017 at 10:32 AM, Anirudh Ramanathan < fox...@google.com.invalid> wrote: > Spark on Kubernetes effort has been developed separately in a fork, and > linked back from the Apache Spark project as an experimental backend >

certification suite?

Hi all, Does anyone happen to know what tests Databricks uses for the Spark distribution certification suite? Is it simply the tests that run as CI on Spark pull requests, or is there something more involved? The web site ( https://databricks.com/spark/certification/certified-spark-distribution)