Yes, see https://dzone.com/articles/predictive-analytics-with-spark-ml
Although the example uses two labels, the same approach supports multiple
labels.
Sent from my iPad
> On Nov 7, 2017, at 6:30 AM, HARSH TAKKAR wrote:
>
> Hi
>
> Does Random Forest in spark Ml supports multi label classi
dition:
>
> ON ((a.col1 = b.col1) or (a.col1 is null and b.col1 is null)) AND ((a.col2 =
> b.col2) or (a.col2 is null and b.col2 is null))
>
> So what we did was re-work our logic to remove the null checks in the join
> condition and the join went lightning fast afterwards :
Good article! Thanks for sharing!
> On Feb 22, 2016, at 11:10 AM, Davies Liu wrote:
>
> This link may help:
> https://forums.databricks.com/questions/6747/how-do-i-get-a-cartesian-product-of-a-huge-dataset.html
>
> Spark 1.6 had improved the CatesianProduct, you should turn of auto
> broadcast
Make sure the xml input file is well formed (check your end tags).
Sent from my iPhone
> On Feb 21, 2016, at 8:14 AM, Prathamesh Dharangutte
> wrote:
>
> This is the code I am using for parsing xml file:
>
>
>
> import org.apache.spark.{SparkConf,SparkContext}
> import org.apache.spark.sq
Try this setting in your Spark defaults:
spark.sql.autoBroadcastJoinThreshold=-1
I had a similar problem with joins hanging and that resolved it for me.
You might be able to pass that value from the driver as a --conf option, but I
have not tried that, and not sure if that will work.
Sent fr
Hi,
We have several udf's written in Scala that we use within jobs submitted into
Spark. They work perfectly with the sqlContext after being registered. We also
allow access to saved tables via the Hive Thrift server bundled with Spark.
However, we would like to allow Hive connections to use th
