Dmitry, Excel-pages:
1). "Compression ratio (2)" - shows object size, with compression and without compression. (Conditions: literal text) 1st graph shows compression ratios of using different compression algrithms depending on size of compressed field. 2nd graph shows evaluation of size of objects depending on sizes and compression algorithms. 2). "Compression ratio (1)" - shows object size, with compression and without compression. (Conditions: badly compressed character sequence) 1st graph shows compression ratios of using different compression algrithms depending on size of compressed field. 2nd graph shows evaluation of size of objects depending on sizes and compression algorithms. 3) 'put-avg" - shows average time of the "put" operation depending on size and compression algorithms. 4) 'put-thrpt" - shows throughput of the "put" operation depending on size and compression algorithms. 5) 'get-avg" - shows average time of the "get" operation depending on size and compression algorithms. 6) 'get-thrpt" - shows throughput of the "get" operation depending on size and compression algorithms. 2017-06-06 10:59 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>: > Vladimir, I am not sure how to interpret the graphs? What are we looking > at? > > On Tue, Jun 6, 2017 at 12:33 AM, Vyacheslav Daradur <daradu...@gmail.com> > wrote: > > > Hi, Igniters. > > > > I've prepared some benchmarking. Results [1]. > > > > And I've prepared the evaluation in the form of diagrams [2]. > > > > I hope that helps to interest the community and accelerates a reaction to > > this improvment :) > > > > [1] > > https://github.com/daradurvs/ignite-compression/tree/ > > master/src/main/resources/result > > [2] https://drive.google.com/file/d/0B2CeUAOgrHkoMklyZ25YTEdKcEk/view > > > > > > > > 2017-05-24 9:49 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > > > > > Guys, any thoughts? > > > > > > 2017-05-16 13:40 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > > > > > >> Hi guys, > > >> > > >> I've prepared the PR to show my idea. > > >> https://github.com/apache/ignite/pull/1951/files > > >> > > >> About querying - I've just copied existing tests and have annotated > the > > >> testing data. > > >> https://github.com/apache/ignite/pull/1951/files#diff-c19a9d > > >> f4058141d059bb577e75244764 > > >> > > >> It means fields which will be marked by @BinaryCompression will be > > >> compressed at marshalling via BinaryMarshaller. > > >> > > >> This solution has no effect on existing data or project architecture. > > >> > > >> I'll be glad to see your thougths. > > >> > > >> > > >> 2017-05-15 19:18 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > > >> > > >>> Dmitriy, > > >>> > > >>> I have ready prototype. I want to show it. > > >>> It is always easier to discuss on example. > > >>> > > >>> 2017-05-15 19:02 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org > >: > > >>> > > >>>> Vyacheslav, > > >>>> > > >>>> I think it is a bit premature to provide a PR without getting a > > >>>> community > > >>>> consensus on the dev list. Please allow some time for the community > to > > >>>> respond. > > >>>> > > >>>> D. > > >>>> > > >>>> On Mon, May 15, 2017 at 6:36 AM, Vyacheslav Daradur < > > >>>> daradu...@gmail.com> > > >>>> wrote: > > >>>> > > >>>> > I created the ticket: https://issues.apache.org/jira > > >>>> /browse/IGNITE-5226 > > >>>> > > > >>>> > I'll prepare a PR with described solution in couple of days. > > >>>> > > > >>>> > 2017-05-15 15:05 GMT+03:00 Vyacheslav Daradur < > daradu...@gmail.com > > >: > > >>>> > > > >>>> > > Hi, Igniters! > > >>>> > > > > >>>> > > Apache 2.0 is released. > > >>>> > > > > >>>> > > Let's continue the discussion about a compression design. > > >>>> > > > > >>>> > > At the moment, I found only one solution which is compatible > with > > >>>> > querying > > >>>> > > and indexing, this is per-objects-field compression. > > >>>> > > Per-fields compression means that metadata (a header) of an > object > > >>>> won't > > >>>> > > be compressed, only serialized values of an object fields (in > > bytes > > >>>> array > > >>>> > > form) will be compressed. > > >>>> > > > > >>>> > > This solution have some contentious issues: > > >>>> > > - small values, like primitives and short arrays - there isn't > > >>>> sense to > > >>>> > > compress them; > > >>>> > > - there is no possible to use compression with java-predefined > > >>>> types; > > >>>> > > > > >>>> > > We can provide an annotation, @IgniteCompression - for example, > > >>>> which can > > >>>> > > be used by users for marking fields to compress. > > >>>> > > > > >>>> > > Any thoughts? > > >>>> > > > > >>>> > > Maybe someone already have ready design? > > >>>> > > > > >>>> > > 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur < > > daradu...@gmail.com > > >>>> >: > > >>>> > > > > >>>> > >> Alexey, > > >>>> > >> > > >>>> > >> Yes, I've read it. > > >>>> > >> > > >>>> > >> Ok, let's discuss about public API design. > > >>>> > >> > > >>>> > >> I think we need to add some a configure entity to > > >>>> CacheConfiguration, > > >>>> > >> which will contain the Compressor interface implementation and > > some > > >>>> > usefull > > >>>> > >> parameters. > > >>>> > >> Or maybe to provide a BinaryMarshaller decorator, which will be > > >>>> compress > > >>>> > >> data after marshalling. > > >>>> > >> > > >>>> > >> > > >>>> > >> 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov < > > akuznet...@apache.org > > >>>> >: > > >>>> > >> > > >>>> > >>> Vyacheslav, > > >>>> > >>> > > >>>> > >>> Did you read initial discussion [1] about compression? > > >>>> > >>> As far as I remember we agreed to add only some "top-level" > API > > in > > >>>> > order > > >>>> > >>> to > > >>>> > >>> provide a way for > > >>>> > >>> Ignite users to inject some sort of custom compression. > > >>>> > >>> > > >>>> > >>> > > >>>> > >>> [1] > > >>>> > >>> http://apache-ignite-developers.2346864.n4.nabble.com/Data-c > > >>>> > >>> ompression-in-Ignite-2-0-td10099.html > > >>>> > >>> > > >>>> > >>> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs < > daradu...@gmail.com > > > > > >>>> > wrote: > > >>>> > >>> > > >>>> > >>> > Hi Igniters! > > >>>> > >>> > > > >>>> > >>> > I am interested in this task. > > >>>> > >>> > Provide some kind of pluggable compression SPI support > > >>>> > >>> > <https://issues.apache.org/jira/browse/IGNITE-3592> > > >>>> > >>> > > > >>>> > >>> > I developed a solution on BinaryMarshaller-level, but > reviewer > > >>>> has > > >>>> > >>> rejected > > >>>> > >>> > it. > > >>>> > >>> > > > >>>> > >>> > Let's continue discussion of task goals and solution design. > > >>>> > >>> > As I understood that, the main goal of this task is to store > > >>>> data in > > >>>> > >>> > compressed form. > > >>>> > >>> > This is what I need from Ignite as its user. Compression > > >>>> provides > > >>>> > >>> economy > > >>>> > >>> > on > > >>>> > >>> > servers. > > >>>> > >>> > We can store more data on same servers at the cost of > > >>>> increasing CPU > > >>>> > >>> > utilization. > > >>>> > >>> > > > >>>> > >>> > I'm researching a possibility of implementation of > compression > > >>>> at the > > >>>> > >>> > cache-level. > > >>>> > >>> > > > >>>> > >>> > Any thoughts? > > >>>> > >>> > > > >>>> > >>> > -- > > >>>> > >>> > Best regards, > > >>>> > >>> > Vyacheslav > > >>>> > >>> > > > >>>> > >>> > > > >>>> > >>> > > > >>>> > >>> > > > >>>> > >>> > -- > > >>>> > >>> > View this message in context: http://apache-ignite- > > >>>> > >>> > developers.2346864.n4.nabble.com/Data-compression-in- > > >>>> > >>> > Ignite-2-0-tp10099p16317.html > > >>>> > >>> > Sent from the Apache Ignite Developers mailing list archive > at > > >>>> > >>> Nabble.com. > > >>>> > >>> > > > >>>> > >>> > > >>>> > >>> > > >>>> > >>> > > >>>> > >>> -- > > >>>> > >>> Alexey Kuznetsov > > >>>> > >>> > > >>>> > >> > > >>>> > >> > > >>>> > >> > > >>>> > >> -- > > >>>> > >> Best Regards, Vyacheslav > > >>>> > >> > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > > -- > > >>>> > > Best Regards, Vyacheslav > > >>>> > > > > >>>> > > > >>>> > > > >>>> > > > >>>> > -- > > >>>> > Best Regards, Vyacheslav > > >>>> > > > >>>> > > >>> > > >>> > > >>> > > >>> -- > > >>> Best Regards, Vyacheslav > > >>> > > >> > > >> > > >> > > >> -- > > >> Best Regards, Vyacheslav > > >> > > > > > > > > > > > > -- > > > Best Regards, Vyacheslav > > > > > > > > > > > -- > > Best Regards, Vyacheslav > > > -- Best Regards, Vyacheslav
