I've already tried UDT in Spark 1.2 and 1.3 but I encountered Kryo Serialization Exception on Joining as tracked here <https://datastax-oss.atlassian.net/browse/SPARKC-23> , i've talked to Michael Armbrust <https://plus.google.com/u/1/109154927192908362223/posts> about the Exception, he said "I'll caution you that this is not a stable public API." So I moved to adding custom dataType into spark. I've got the answer of my this question from Iulian Dragoș <https://plus.google.com/u/1/114220203404330592389/posts> that "One way is to use export SPARK_PREPEND_CLASSES=true. This will instruct the launcher to prepend the target directories for each project to the spark assembly", this solved my problem. Thanks...
-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Building-Spark-Adding-new-DataType-in-Catalyst-tp22604p22632.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
