Re: A proposal for Spark 2.0

2015-11-11 Thread hitoshi
It looks like Chill is willing to upgrade their Kryo to 3.x if Spark and Hive
will. As it is now Spark, Chill, and Hive have Kryo jar but it really can't
be used because Kryo 2 can't serdes some classes. Since Spark 2.0 is a major
release, it really would be nice if we can resolve the Kryo issue.
 
https://github.com/twitter/chill/pull/230#issuecomment-155845959



--
View this message in context: 
http://apache-spark-developers-list.1001551.n3.nabble.com/A-proposal-for-Spark-2-0-tp15122p15163.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org



Re: A proposal for Spark 2.0

2015-11-11 Thread hitoshi
Resending my earlier message because it wasn't accepted.

Would like to add a proposal to upgrade jars when they do not break APIs and
fixes a bug. 
To be more specific, I would like to see Kryo to be upgraded from 2.21 to
3.x. Kryo 2.x has a bug (e.g.SPARK-7708) that is blocking it usage in
production environment. 
Other projects like Chill is also wanting to upgrade Kryo to 3.x but being
blocked because Spark won't upgrade. I think OSS community at large will
benefit if we can coordinate to upgrade to Kryo 3.x 



--
View this message in context: 
http://apache-spark-developers-list.1001551.n3.nabble.com/A-proposal-for-Spark-2-0-tp15122p15164.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org