custom REST port from spark-defaults.cof

Hi, is there a configuration setting to set a custom port number for the master REST URL? can that be included in the spark-defaults.conf? cheers -- Niranda @n1r44 https://pythagoreanscript.wordpress.com/

Force Spark save parquet files with replication factor other than 3 (default one)

Hi, My Hadoop is configured to have replication ratio = 2. I've added $HADOOP_HOME/config to the PATH as suggested in http://apache-spark-user-list.1001560.n3.nabble.com/hdfs-replication-on-saving-RDD-td289.html. Spark (1.4) does rdd.saveAsTextFile with replication=2. However DataFrame.saveAsP

Re: Hive 0.12 support in 1.4.0 ?

Hi Tom, In Spark 1.4, we have de-coupled the support of Hive's metastore and other parts (parser, Hive udfs, and Hive SerDes). The execution engine of Spark SQL in 1.4 will always use Hive 0.13.1. For the metastore connection part, you can connect to either Hive 0.12 or 0.13.1's metastore. We have