Dmitriy, I would suggest us not limiting blobs use case to a dedicated cache. If to look at other databases they usually have BLOB/LONGBLOB/etc. as a type meaning that users mix simple and BLOB types in the same tables.
Should we start with Ignite SQL adding blobs through its APIs? -- Denis On Fri, Aug 3, 2018 at 1:52 PM Dmitriy Setrakyan <dsetrak...@apache.org> wrote: > Pavel, > > Not everything that gets put in Ignite has a class, and not everything can > be annotated. For example, what about byte[] or a binary object? > > I would prefer a separate cache with specific purpose of storing blobs. It > will be easier for users to understand and configure. It can also have a > custom logic of splitting a blob into multiple batches to be able to > transfer it over the network faster. > > D. > > > On Fri, Aug 3, 2018 at 3:21 AM, Pavel Kovalenko <jokse...@gmail.com> > wrote: > > > Dmitriy, > > > > I think we don't need a separate implementation of cache like BLOB cache. > > Instead of it, a user can mark value class or value class field with the > > special annotation "@BLOB". > > During cache put, marshaller will place a special placeholder on such > > fields, write byte[] array payload of a field to special internal blob > > storage and place the only reference to actual DataEntry in the page > > memory. > > During cache get, marshaller will place a special proxy instead of an > > actual class that can be downloaded and unmarshalled by demand from > > internal storage on the user side. > > Using such approach we will also solve eager/lazy problem, this will also > > give user possibility to adjust his own marshallers (like Jackson, Jaxb, > > etc.) to marshal/unmarshal his blob classes from/to byte[] arrays. > > No major changes in public API are required, it can be pluggable > component. > > > > > > 2018-08-03 0:25 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>: > > > > > On Thu, Aug 2, 2018 at 1:08 AM, Pavel Kovalenko <jokse...@gmail.com> > > > wrote: > > > > > > > Dmitriy, > > > > > > > > I still don't understand why do you think that it will be file > system? > > > > In all my previous messages I emphasized that this storage shouldn't > be > > > > considered as a file system. It's just a large data storage, whose > > > entities > > > > can be easily accessed using key/link (internally, or externally > using > > > > web/binary protocol interfaces). > > > > > > > > > Instead, if we must focus on large blobs, I would solve the problem > > of > > > > supporting large blobs in regular Ignite caches, as I suggested > before. > > > > > > > > This is impossible. Our page memory can't handle efficiently it by > > > design. > > > > > > > > > > But our API does. What is stopping us from creating a cache as a BLOB > > cache > > > and using whatever storage we need? > > > > > >
