I’ve been thinking about how messages are stored in the broker and ways to improve the storage in memory.
First, right now, messages are stored in the same heap, and if you’re using the memory store, like, that’s going to add up. This will increase GC latency , and you actually need 2x more memory because you have to have temp memory set aside for GCs. I was thinking about using Chronicle to store the messages off heap using direct buffers. The downside to this is that the messages need to be serialized/deserialized with each access. But realistically that’s probably acceptable because you can do something like 1M message deserializations per second. Which is normally more than the throughput of the broker. Additionally, chronicle supports zlib or snappy compression on the message bodies. So, while the broker supports message compression now, it doesn’t support this feature on headers. This would give us header compression! The broker would transparently decompress the headers when reading the message. This then begs the question, why use message bodies at all? Why not just store an entire message as a set of headers? If you need hierarchy you can do foo.bar.cat.dog style header names. -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts> <http://spinn3r.com>
