There was work being done at Berkeley on prototyping support for Succinct in Spark SQL. Rachit might have more information.
On Thu, Jan 22, 2015 at 7:04 AM, Dean Wampler <deanwamp...@gmail.com> wrote: > Interesting. I was wondering recently if anyone has explored working with > compressed data directly. > > Dean Wampler, Ph.D. > Author: Programming Scala, 2nd Edition > <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) > Typesafe <http://typesafe.com> > @deanwampler <http://twitter.com/deanwampler> > http://polyglotprogramming.com > > On Thu, Jan 22, 2015 at 2:59 AM, Mick Davies <michael.belldav...@gmail.com > > > wrote: > > > > > http://succinct.cs.berkeley.edu/wp/wordpress/ > > > > Looks like a really interesting piece of work that could dovetail well > with > > Spark. > > > > I have been trying recently to optimize some queries I have running on > > Spark > > on top of Parquet but the support from Parquet for predicate push down > > especially for dictionary based columns is a bit limiting. I am not sure, > > but from a cursory view it looks like this format may help in this area. > > > > Mick > > > > > > > > > > -- > > View this message in context: > > > http://apache-spark-developers-list.1001551.n3.nabble.com/Are-there-any-plans-to-run-Spark-on-top-of-Succinct-tp10243.html > > Sent from the Apache Spark Developers List mailing list archive at > > Nabble.com. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > > For additional commands, e-mail: dev-h...@spark.apache.org > > > > >
