Hi Alex, Thanks for your input (and all your work on Carbonite)- much appreciated!
1) buffer reuse - reusing cached ThreadLocal byte[] for converting from > data to bytes is a nice trick and made possible in most serialization libs > but I don't that's possible in the current code. > I experimented with a number of different buffering approaches, including a reusable ByteBuffer. Didn't see as big of a performance boost as I was expecting, so elected to keep it simple for the moment. It's very possible I missed something or tested incorrectly: I'll certainly check out your approach! > 2) transients while deserializing collections (in coll-thaw!) - I found > this to be the only way to get anywhere close to Java serialization while > benchmarking. > I'm relying on `into` when possible, which is using transients under-the-hood. The notable exception is for maps - but that was because the only approaches I could think of that'd allow transients actually turned out to be slower in practice due to other overhead. A transient-based zipmap was the best perf I could get, but it was only ~10% faster and it incurred some other effects I didn't like. 3) more efficient primitive writing - this is the main reason I wrapped > Kryo - there are lots of well-known tricks for writing chars, ints, etc. > Thrift, Protobuff, Kryo, and others have made these pretty standard and > they make a huge difference vs the basic DataInputStream serialization. > This I'm completely unfamiliar with and it sounds interesting: I'll definitely check it out as well, thank you! > I found the use of transients in deserializing collections to be the only > way I could get anywhere close to Java serialization performance - they are > critical. > To be honest, performance for Nippy was actually a relatively low priority for me since in its current/obvious form, it's already fast enough that it ceases to be the bottleneck in any practical contexts in which I would be using it (mostly DB-related). That's to say, I haven't spent an awful lot of effort tweaking for performance and there's probably lots of low-hanging fruit. I'll start by checking out what you've suggested, and I'm very open to input if you (or anyone else) thinks of other ideas! Cheers! -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
