Hi For approach 1: Put a large object into a partiton cache will force to update the dictionary placed on replication cache. It seeis it may be time-expense operation. Appoach 2-3 are make sense for rare cases as Sergi commented. Aslo I see a danger of OOM if we've got high compression level and try to restore orginal value in memory.
On Mon, Jul 25, 2016 at 10:39 AM, Alexey Kuznetsov <akuznet...@gridgain.com> wrote: > Sergi, > > Of course it will introduce some slowdown, but with compression more data > could be stored in memory > and not will be evicted to disk. In case of compress by dictionary > substitution it will be only one more lookup > and should be fast. > > In general we could provide only API for compression out of the box, and > users that really need some sort of compression > will implement it by them self. This will not require much effort I think. > > > > On Mon, Jul 25, 2016 at 2:18 PM, Sergi Vladykin <sergi.vlady...@gmail.com> > wrote: > > > This will make sense only for rare cases when you have very large objects > > stored, which can be effectively compressed. And even then it will > > introduce slowdown on all the operations, which often will not be > > acceptable. I guess only few users will find this feature useful, thus I > > think it does not worth the effort. > > > > Sergi > > > > > > > > 2016-07-25 9:28 GMT+03:00 Alexey Kuznetsov <akuznet...@gridgain.com>: > > > > > Hi, All! > > > > > > I would like to propose one more feature for Ignite 2.0. > > > > > > Data compression for data in binary format. > > > > > > Binary format is stored as field name + field data. > > > So we have a description. > > > How about to add one more byte to binary data descriptor: > > > > > > *Compressed*: > > > 0 - Data stored as is (no compression). > > > 1 - Data compressed by dictionary (something like DB2 row compression > > [1], > > > but for all binary types). We could have system or user defined > > replicated > > > cache for such dictionary and *cache.compact()* method that will scan > > > cache, build dictionary and compact data. > > > 2 - Data compressed by Java built in ZIP. > > > 3 - Data compressed by some user custom algorithm. > > > > > > Of course it is possible to compress data in current Ignite 1.x but in > > this > > > case compressed data cannot be accessed from SQL engine, if we > implement > > > support for compression on Ignite core level SQL engine will be able to > > > detect that data is compressed and properly handle such data. > > > > > > What do you think? > > > If community consider this feature useful I will create issue in JIRA. > > > > > > [1] > > > > > > > > > http://www.ibm.com/developerworks/data/library/techarticle/dm-1205db210compression/ > > > > > > -- > > > Alexey Kuznetsov > > > > > > > > > -- > Alexey Kuznetsov > -- Sergey Kozlov GridGain Systems www.gridgain.com
