Hi, Igniters! Apache 2.0 is released.
Let's continue the discussion about a compression design. At the moment, I found only one solution which is compatible with querying and indexing, this is per-objects-field compression. Per-fields compression means that metadata (a header) of an object won't be compressed, only serialized values of an object fields (in bytes array form) will be compressed. This solution have some contentious issues: - small values, like primitives and short arrays - there isn't sense to compress them; - there is no possible to use compression with java-predefined types; We can provide an annotation, @IgniteCompression - for example, which can be used by users for marking fields to compress. Any thoughts? Maybe someone already have ready design? 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > Alexey, > > Yes, I've read it. > > Ok, let's discuss about public API design. > > I think we need to add some a configure entity to CacheConfiguration, > which will contain the Compressor interface implementation and some usefull > parameters. > Or maybe to provide a BinaryMarshaller decorator, which will be compress > data after marshalling. > > > 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov <akuznet...@apache.org>: > >> Vyacheslav, >> >> Did you read initial discussion [1] about compression? >> As far as I remember we agreed to add only some "top-level" API in order >> to >> provide a way for >> Ignite users to inject some sort of custom compression. >> >> >> [1] >> http://apache-ignite-developers.2346864.n4.nabble.com/Data- >> compression-in-Ignite-2-0-td10099.html >> >> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs <daradu...@gmail.com> wrote: >> >> > Hi Igniters! >> > >> > I am interested in this task. >> > Provide some kind of pluggable compression SPI support >> > <https://issues.apache.org/jira/browse/IGNITE-3592> >> > >> > I developed a solution on BinaryMarshaller-level, but reviewer has >> rejected >> > it. >> > >> > Let's continue discussion of task goals and solution design. >> > As I understood that, the main goal of this task is to store data in >> > compressed form. >> > This is what I need from Ignite as its user. Compression provides >> economy >> > on >> > servers. >> > We can store more data on same servers at the cost of increasing CPU >> > utilization. >> > >> > I'm researching a possibility of implementation of compression at the >> > cache-level. >> > >> > Any thoughts? >> > >> > -- >> > Best regards, >> > Vyacheslav >> > >> > >> > >> > >> > -- >> > View this message in context: http://apache-ignite- >> > developers.2346864.n4.nabble.com/Data-compression-in- >> > Ignite-2-0-tp10099p16317.html >> > Sent from the Apache Ignite Developers mailing list archive at >> Nabble.com. >> > >> >> >> >> -- >> Alexey Kuznetsov >> > > > > -- > Best Regards, Vyacheslav > -- Best Regards, Vyacheslav
