ByteBuffer is not fundamental data type. It is a thin wrapper over array. Protocol should contain only fundamental types.
вт, 7 нояб. 2017 г. в 10:05, Andrey Kornev <andrewkor...@hotmail.com>: > Vladimir, > > Could you please elaborate as to why adding ByteBuffer to the Binary > protocol is “not a good idea”? What is inherently wrong about it? > > As for your suggestion, while I certainly see your point, it comes with > the inconvenience of implementing Binarylizable for every entity type in an > application, which makes me wonder if the self imposed rules such as “type > system should be kept as small as possible” could be relaxed in the name of > greater good and user friendliness. Ignite API is not easy to grok as it is > already. > > Thanks > Andrey > _____________________________ > From: Vladimir Ozerov <voze...@gridgain.com<mailto:voze...@gridgain.com>> > Sent: Monday, November 6, 2017 10:22 PM > Subject: Re: Deserialization of BinaryObject's byte arrays > To: <dev@ignite.apache.org<mailto:dev@ignite.apache.org>> > Cc: Valentin Kulichenko <valentin.kuliche...@gmail.com<mailto: > valentin.kuliche...@gmail.com>> > > > Hi Andrey, > > While we deifnitely need to support byte buffers, adding new type to > protocol is not very good idea. Binary protocol is the heart of our ssytem, > which is used not oly for plain Java, but for other platforms and SQL. And > ByteBuffer is essentially the same byte array, but with different API. What > you describe is a special case of byte array read/write. For this reason > our type system should be kept as small as possible and it doesn't require > new type to be introduced. Instead, we'd better to handle everything on API > level only. For example, we can add new methods to reader/writer/builder, > which will handle this. E.g.: > > BinaryWriter.writeByteArray(<some producer here>); > BinaryReader.readByteArray(<some consumer here>); > > Vladimir. > > On Mon, Nov 6, 2017 at 9:25 PM, Andrey Kornev <andrewkor...@hotmail.com > <mailto:andrewkor...@hotmail.com>> > wrote: > > > Will do! Thanks! > > ________________________________ > > From: Valentin Kulichenko <valentin.kuliche...@gmail.com<mailto: > valentin.kuliche...@gmail.com>> > > Sent: Monday, November 6, 2017 9:17 AM > > To: dev@ignite.apache.org<mailto:dev@ignite.apache.org> > > Subject: Re: Deserialization of BinaryObject's byte arrays > > > > This use case was discussed couple of times already, so it seems to be > > pretty important for users. And I like the approach suggested by Andrey. > > > > Andrey, can you create a Jira ticket for this change? Would you like to > > contribute it? > > > > -Val > > > > On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan <dsetrak...@apache.org > <mailto:dsetrak...@apache.org>> > > wrote: > > > > > This makes sense to me. Sounds like a useful feature. > > > > > > Would be nice to hear what others in the community think? > > > > > > D. > > > > > > On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev < > andrewkor...@hotmail.com<mailto:andrewkor...@hotmail.com>> > > > wrote: > > > > > > > Hello, > > > > > > > > We store lots of byte arrays (serialized graph-like data structures) > as > > > > fields of BinaryObject. Later on, while reading such field, > > > > BinaryInputStream implementation creates an on-heap array and copies > > the > > > > bytes from the BinaryObject's internal backing array to the new > array. > > > > > > > > While in general case it works just fine, in our case, a lot of CPU > is > > > > spent on copying of the bytes and significant amount garbage is > > generated > > > > as well. > > > > > > > > I'd like Ignite Community to consider the following proposal: > introduce > > > > support for serialization/deserialization of the ByteBuffer class. A > > > > BinaryInputStream implementation would special case ByteBuffer fields > > > same > > > > way it currently special cases various Java types (collections, > dates, > > > > timestamps, enums, etc.) > > > > > > > > The application developer would simply call > > > BinaryObjectBuilder.setField(...) > > > > passing in an instance of ByteBuffer. During serialization, the > > > > ByteBuffer's bytes would simply be copied to the backing array > (similar > > > to > > > > how regular byte arrays are serialized today) and in the binary > > object's > > > > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER. > > > > > > > > During deserialization of the field, instead of allocating a new byte > > > > array and transferring the bytes from the BinaryObject's backing > array, > > > > BinaryInputStream would simply wrap the required sub-array into a > > > read-only > > > > instance of ByteBuffer using "public static ByteBuffer wrap(byte[] > > array, > > > > int offset, int length)" and return the instance to the application. > > > > > > > > This approach doesn't require any array allocation/data copying and > is > > > > faster and significantly reduces pressure on the garbage collector. > The > > > > benefits are especially great when application stores significant > > amount > > > of > > > > its data as byte arrays. > > > > > > > > Clearly, this approach equally applies to arrays of other primitive > > > types. > > > > > > > > A minor complication does arise when dealing with off-heap > > BinaryObjects, > > > > but nothing that can't solved with a little bit of Java reflection > > > wizardry > > > > (or by other means). > > > > > > > > Thanks, > > > > Andrey > > > > > > > > > > > >
