It is actually pluggable. You can implement new compression codecs and just change the config variable to use those.
On Tuesday, June 17, 2014, gchen <chenguanch...@gmail.com> wrote: > Cool, so maybe when we swith to Snappy instead of LZF, we can workaround > the > bug until the LZF upstream fix it, right? > > In addition, is it valuable to add support for other compression codecs > such > as LZ4? We observed 5% end-to-end improvement using LZ4 vs Snappy in > Terasort (Hadoop MR). > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/Big-Endian-IBM-Power7-Spark-Serialization-issue-tp7003p7037.html > Sent from the Apache Spark Developers List mailing list archive at > Nabble.com. >
