Will do! Thanks! ________________________________ From: Valentin Kulichenko <valentin.kuliche...@gmail.com> Sent: Monday, November 6, 2017 9:17 AM To: dev@ignite.apache.org Subject: Re: Deserialization of BinaryObject's byte arrays
This use case was discussed couple of times already, so it seems to be pretty important for users. And I like the approach suggested by Andrey. Andrey, can you create a Jira ticket for this change? Would you like to contribute it? -Val On Fri, Nov 3, 2017 at 4:18 PM, Dmitriy Setrakyan <dsetrak...@apache.org> wrote: > This makes sense to me. Sounds like a useful feature. > > Would be nice to hear what others in the community think? > > D. > > On Fri, Nov 3, 2017 at 1:07 PM, Andrey Kornev <andrewkor...@hotmail.com> > wrote: > > > Hello, > > > > We store lots of byte arrays (serialized graph-like data structures) as > > fields of BinaryObject. Later on, while reading such field, > > BinaryInputStream implementation creates an on-heap array and copies the > > bytes from the BinaryObject's internal backing array to the new array. > > > > While in general case it works just fine, in our case, a lot of CPU is > > spent on copying of the bytes and significant amount garbage is generated > > as well. > > > > I'd like Ignite Community to consider the following proposal: introduce > > support for serialization/deserialization of the ByteBuffer class. A > > BinaryInputStream implementation would special case ByteBuffer fields > same > > way it currently special cases various Java types (collections, dates, > > timestamps, enums, etc.) > > > > The application developer would simply call > BinaryObjectBuilder.setField(...) > > passing in an instance of ByteBuffer. During serialization, the > > ByteBuffer's bytes would simply be copied to the backing array (similar > to > > how regular byte arrays are serialized today) and in the binary object's > > metadata the field marked as, say, GridBinaryMarshaller.BYTE_BUFFER. > > > > During deserialization of the field, instead of allocating a new byte > > array and transferring the bytes from the BinaryObject's backing array, > > BinaryInputStream would simply wrap the required sub-array into a > read-only > > instance of ByteBuffer using "public static ByteBuffer wrap(byte[] array, > > int offset, int length)" and return the instance to the application. > > > > This approach doesn't require any array allocation/data copying and is > > faster and significantly reduces pressure on the garbage collector. The > > benefits are especially great when application stores significant amount > of > > its data as byte arrays. > > > > Clearly, this approach equally applies to arrays of other primitive > types. > > > > A minor complication does arise when dealing with off-heap BinaryObjects, > > but nothing that can't solved with a little bit of Java reflection > wizardry > > (or by other means). > > > > Thanks, > > Andrey > > >
