Yes, there's no such thing as writing a deserialized form to disk. However there are other persistence levels that store *serialized* forms in memory. The meaning here is that the objects are not serialized in memory in the JVM. Of course, they are serialized on disk.
On Sun, Aug 31, 2014 at 5:02 AM, Tao Xiao <xiaotao.cs....@gmail.com> wrote: > Reading about RDD Persistency, I learned that the storage level > "MEMORY_AND_DISK" means that " Store RDD as deserialized Java objects in the > JVM. If the RDD does not fit in memory, store the partitions that don't fit > on disk, and read them from there when they're needed. " > > But how can a "deserialized Java object" be stored on disk? As far as I > know, a Java object should be stored as an array of bytes on disk, which > means that Java object should be firtly converted into an array of bytes (a > serialized object). > > Thanks . --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
