Re: Size of Java serialized Clojure data structures

2015-08-08 Thread icamts
Hi Alan, I did the same experiment, with the same variable names too :) have a look at the github issue. Finally I discovered prevayler calls reset method on oos every time it writes to it. This is why the overhead is written every time. Klaus, prevayler author, says it is so to prevent leaks an

Re: Size of Java serialized Clojure data structures

2015-08-07 Thread Alan Malloy
:a1, but it is still just 80 bytes. Nowhere near 1KB per small vector. On Friday, August 7, 2015 at 6:31:46 AM UTC-7, icamts wrote: > > Yes. I suggested nippy. The question is about the size of Java serialized > Clojure data structures. Can a two element vector be 1kB in size? Why > se

Re: Size of Java serialized Clojure data structures

2015-08-07 Thread icamts
Yes. I suggested nippy. The question is about the size of Java serialized Clojure data structures. Can a two element vector be 1kB in size? Why serialization in my REPL experiment (see the code following the link in my previous mail) produces a 80MB byte buffer while prevayler logs are 1GB? Il

Re: Size of Java serialized Clojure data structures

2015-08-07 Thread Gary Verhaegen
You should probably look at Clojure-specific solutions, starting with EDN (i.e. essentially pr-str), fressian, transit or nippy. Clojure data structures have a lot of properties that can be exploited (we only care about the abstract type and the actual data), so a serializer can make a lot of assu

Size of Java serialized Clojure data structures

2015-08-07 Thread icamts
Hi all, I opened this issue on github project prevayler-clj https://github.com/klauswuestefeld/prevayler-clj/issues/1 because 1M short vectors, like this [:a1 1], forming the state of the prevayler, results in 1GB file size when serialized, one by one, with Java writeObject. Is it possible? Ab