>> To make it fair. Ignite uses thread-local reusable buffers, see [1]. I know, but PooledMessageBufferOutput is not about thread-local, isn't it?
I'm not against about MsgPack, I'm for fair and not biased comparison. I suppose that MsgPack is an ideal candidate for thin client binary protocol, not only for serializing some user data. I am analyzed some tarantool connectors [1] [2] [3] and found msgpack based protocol is a really good idea. Also there is realy super fast and just 1 header library with liberal BSD-2 licence for C -- msgpuck [4]. It used in tarantool itself and in [1] and is stable and bullet proof. [1] -- https://github.com/igorcoding/asynctnt [2] -- https://github.com/tarantool/tarantool-python/ [3] -- https://github.com/tarantool/go-tarantool [4] -- https://github.com/rtsisyk/msgpuck пт, 18 июн. 2021 г. в 11:44, Pavel Tupitsyn <ptupit...@apache.org>: > Ivan, > > > why do you use PooledMessageBufferOutput in benchmarks? > > To make it fair. Ignite uses thread-local reusable buffers, see [1]. > > > > why packer from msgpack-core show better performance than > > BinaryWriter. And I suppose that benchmark is not quite fair. > > MsgPack writes and reads less bytes, so it should be faster. > Benchmark is not 100% fair, there are some small extra things that > BinaryWriter does. > > However: > 1. I don't think we care about super-precise benchmarks here, just the > ballpark. > 2. We are discussing the format, not the implementation. > > Important takeaway is: > The format does not prevent someone from implementing it efficiently. > > > > [1] > > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryWriterExImpl.java#L101 > > On Thu, Jun 17, 2021 at 10:40 PM Ivan Daschinsky <ivanda...@gmail.com> > wrote: > > > Andrey, here we discuss serialization format, as far as I understand. > > Current implementation of ignite binary object serialization can be > > rewritten. > > If we do not care about fast (O(1)) field lookup, about schema validation > > and so on, msgpack is a really good option. It is also good for client > > binary protocol, i.e. > > tarantool uses it. > > > > >> Binarilizable interface forces user to write serialization code > > I am talking about speed comparison. You can see from Pavel's data, > > jackson-msgpack shows a pathetic performance comparing with a ignite's > > default binary marshaller. If you want really fast serialization -- the > > only option is to write code by yourself or use code generation. Default > > packer from msgpack-core java package is similar to BinaryWriter. So I am > > wondering why packer from msgpack-core show better performance than > > BinaryWriter. And I suppose that benchmark is not quite fair. > > > > > > чт, 17 июн. 2021 г. в 22:19, Andrey Mashenkov < > andrey.mashen...@gmail.com > > >: > > > > > Ivan, thankd for clarification. > > > > > > Binarilizable interface forces user to write serialization code. We can > > > support this or similar interface. > > > But I'd like Ignite has some default serializer in addition. It can be > > also > > > useful e.g. in compute for param and result serialization. > > > > > > BinaryObjectBuider requires an Ignite node for object construction, but > > we > > > are looking for a detached builder and won't care about schemas. > > > > > > AFAIR, BinaryObject creates an objectReader on every single field read > > > operation. > > > So, BO solution produces a lot of garbage and BO has noticable overhead > > > which affects the object footprint. > > > > > > чт, 17 июн. 2021 г., 21:41 Ivan Daschinsky <ivanda...@gmail.com>: > > > > > > > >> Double checked -- there is not any links to PR either in IEP or in > > > jira > > > > issue > > > > Sorry, there is a link in IEP, but not in jira ticket. > > > > > > > > чт, 17 июн. 2021 г. в 21:39, Ivan Daschinsky <ivanda...@gmail.com>: > > > > > > > > > Andrey, > > > > > >> arbitrary object graph > > > > > Also, that is not true, msgpack format doesn't handle circular > > graphs. > > > > > Think about msgpack as binary json. You couldn't understand full > > > > structure > > > > > of message if you didn't deserialize it fully before, maps and > arrays > > > are > > > > > serialized just as contiguos chunks > > > > > of values/kv-pairs. Msgpack is a really dumb and simple format. > > > > > > > > > > Also, as for me, I cannot understand why current ignite > serialization > > > > > (BinaryObjectBuilder or Binarilizable) is slower than raw message > > pack > > > > > serializer. > > > > > I suppose that this is an issue and we should investigate it. > > > > > > > > > > Pavel, why do you use PooledMessageBufferOutput in benchmarks? > I'm > > > > > sorry, but is it fair to use it? > > > > > > > > > > >> The code is linked in the IEP [2] > > > > > Double checked -- there is not any links to PR either in IEP or in > > jira > > > > > issue > > > > > > > > > > > > > > > > > -- > > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > -- > > Sincerely yours, Ivan Daschinskiy > > > -- Sincerely yours, Ivan Daschinskiy
