All metrics are taken from app based on custom assembly of AI, containing the provided PR.
2017-06-06 11:28 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > Dmitry, > > Excel-pages: > > 1). "Compression ratio (2)" - shows object size, with compression and > without compression. (Conditions: literal text) > 1st graph shows compression ratios of using different compression > algrithms depending on size of compressed field. > 2nd graph shows evaluation of size of objects depending on sizes and > compression algorithms. > > 2). "Compression ratio (1)" - shows object size, with compression and > without compression. (Conditions: badly compressed character sequence) > 1st graph shows compression ratios of using different compression > algrithms depending on size of compressed field. > 2nd graph shows evaluation of size of objects depending on sizes and > compression algorithms. > > 3) 'put-avg" - shows average time of the "put" operation depending on size > and compression algorithms. > > 4) 'put-thrpt" - shows throughput of the "put" operation depending on size > and compression algorithms. > > 5) 'get-avg" - shows average time of the "get" operation depending on size > and compression algorithms. > > 6) 'get-thrpt" - shows throughput of the "get" operation depending on size > and compression algorithms. > > > > > 2017-06-06 10:59 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>: > >> Vladimir, I am not sure how to interpret the graphs? What are we looking >> at? >> >> On Tue, Jun 6, 2017 at 12:33 AM, Vyacheslav Daradur <daradu...@gmail.com> >> wrote: >> >> > Hi, Igniters. >> > >> > I've prepared some benchmarking. Results [1]. >> > >> > And I've prepared the evaluation in the form of diagrams [2]. >> > >> > I hope that helps to interest the community and accelerates a reaction >> to >> > this improvment :) >> > >> > [1] >> > https://github.com/daradurvs/ignite-compression/tree/ >> > master/src/main/resources/result >> > [2] https://drive.google.com/file/d/0B2CeUAOgrHkoMklyZ25YTEdKcEk/view >> > >> > >> > >> > 2017-05-24 9:49 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: >> > >> > > Guys, any thoughts? >> > > >> > > 2017-05-16 13:40 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: >> > > >> > >> Hi guys, >> > >> >> > >> I've prepared the PR to show my idea. >> > >> https://github.com/apache/ignite/pull/1951/files >> > >> >> > >> About querying - I've just copied existing tests and have annotated >> the >> > >> testing data. >> > >> https://github.com/apache/ignite/pull/1951/files#diff-c19a9d >> > >> f4058141d059bb577e75244764 >> > >> >> > >> It means fields which will be marked by @BinaryCompression will be >> > >> compressed at marshalling via BinaryMarshaller. >> > >> >> > >> This solution has no effect on existing data or project architecture. >> > >> >> > >> I'll be glad to see your thougths. >> > >> >> > >> >> > >> 2017-05-15 19:18 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: >> > >> >> > >>> Dmitriy, >> > >>> >> > >>> I have ready prototype. I want to show it. >> > >>> It is always easier to discuss on example. >> > >>> >> > >>> 2017-05-15 19:02 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org >> >: >> > >>> >> > >>>> Vyacheslav, >> > >>>> >> > >>>> I think it is a bit premature to provide a PR without getting a >> > >>>> community >> > >>>> consensus on the dev list. Please allow some time for the >> community to >> > >>>> respond. >> > >>>> >> > >>>> D. >> > >>>> >> > >>>> On Mon, May 15, 2017 at 6:36 AM, Vyacheslav Daradur < >> > >>>> daradu...@gmail.com> >> > >>>> wrote: >> > >>>> >> > >>>> > I created the ticket: https://issues.apache.org/jira >> > >>>> /browse/IGNITE-5226 >> > >>>> > >> > >>>> > I'll prepare a PR with described solution in couple of days. >> > >>>> > >> > >>>> > 2017-05-15 15:05 GMT+03:00 Vyacheslav Daradur < >> daradu...@gmail.com >> > >: >> > >>>> > >> > >>>> > > Hi, Igniters! >> > >>>> > > >> > >>>> > > Apache 2.0 is released. >> > >>>> > > >> > >>>> > > Let's continue the discussion about a compression design. >> > >>>> > > >> > >>>> > > At the moment, I found only one solution which is compatible >> with >> > >>>> > querying >> > >>>> > > and indexing, this is per-objects-field compression. >> > >>>> > > Per-fields compression means that metadata (a header) of an >> object >> > >>>> won't >> > >>>> > > be compressed, only serialized values of an object fields (in >> > bytes >> > >>>> array >> > >>>> > > form) will be compressed. >> > >>>> > > >> > >>>> > > This solution have some contentious issues: >> > >>>> > > - small values, like primitives and short arrays - there isn't >> > >>>> sense to >> > >>>> > > compress them; >> > >>>> > > - there is no possible to use compression with java-predefined >> > >>>> types; >> > >>>> > > >> > >>>> > > We can provide an annotation, @IgniteCompression - for example, >> > >>>> which can >> > >>>> > > be used by users for marking fields to compress. >> > >>>> > > >> > >>>> > > Any thoughts? >> > >>>> > > >> > >>>> > > Maybe someone already have ready design? >> > >>>> > > >> > >>>> > > 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur < >> > daradu...@gmail.com >> > >>>> >: >> > >>>> > > >> > >>>> > >> Alexey, >> > >>>> > >> >> > >>>> > >> Yes, I've read it. >> > >>>> > >> >> > >>>> > >> Ok, let's discuss about public API design. >> > >>>> > >> >> > >>>> > >> I think we need to add some a configure entity to >> > >>>> CacheConfiguration, >> > >>>> > >> which will contain the Compressor interface implementation and >> > some >> > >>>> > usefull >> > >>>> > >> parameters. >> > >>>> > >> Or maybe to provide a BinaryMarshaller decorator, which will >> be >> > >>>> compress >> > >>>> > >> data after marshalling. >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov < >> > akuznet...@apache.org >> > >>>> >: >> > >>>> > >> >> > >>>> > >>> Vyacheslav, >> > >>>> > >>> >> > >>>> > >>> Did you read initial discussion [1] about compression? >> > >>>> > >>> As far as I remember we agreed to add only some "top-level" >> API >> > in >> > >>>> > order >> > >>>> > >>> to >> > >>>> > >>> provide a way for >> > >>>> > >>> Ignite users to inject some sort of custom compression. >> > >>>> > >>> >> > >>>> > >>> >> > >>>> > >>> [1] >> > >>>> > >>> http://apache-ignite-developers.2346864.n4.nabble.com/Data-c >> > >>>> > >>> ompression-in-Ignite-2-0-td10099.html >> > >>>> > >>> >> > >>>> > >>> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs < >> daradu...@gmail.com >> > > >> > >>>> > wrote: >> > >>>> > >>> >> > >>>> > >>> > Hi Igniters! >> > >>>> > >>> > >> > >>>> > >>> > I am interested in this task. >> > >>>> > >>> > Provide some kind of pluggable compression SPI support >> > >>>> > >>> > <https://issues.apache.org/jira/browse/IGNITE-3592> >> > >>>> > >>> > >> > >>>> > >>> > I developed a solution on BinaryMarshaller-level, but >> reviewer >> > >>>> has >> > >>>> > >>> rejected >> > >>>> > >>> > it. >> > >>>> > >>> > >> > >>>> > >>> > Let's continue discussion of task goals and solution >> design. >> > >>>> > >>> > As I understood that, the main goal of this task is to >> store >> > >>>> data in >> > >>>> > >>> > compressed form. >> > >>>> > >>> > This is what I need from Ignite as its user. Compression >> > >>>> provides >> > >>>> > >>> economy >> > >>>> > >>> > on >> > >>>> > >>> > servers. >> > >>>> > >>> > We can store more data on same servers at the cost of >> > >>>> increasing CPU >> > >>>> > >>> > utilization. >> > >>>> > >>> > >> > >>>> > >>> > I'm researching a possibility of implementation of >> compression >> > >>>> at the >> > >>>> > >>> > cache-level. >> > >>>> > >>> > >> > >>>> > >>> > Any thoughts? >> > >>>> > >>> > >> > >>>> > >>> > -- >> > >>>> > >>> > Best regards, >> > >>>> > >>> > Vyacheslav >> > >>>> > >>> > >> > >>>> > >>> > >> > >>>> > >>> > >> > >>>> > >>> > >> > >>>> > >>> > -- >> > >>>> > >>> > View this message in context: http://apache-ignite- >> > >>>> > >>> > developers.2346864.n4.nabble.com/Data-compression-in- >> > >>>> > >>> > Ignite-2-0-tp10099p16317.html >> > >>>> > >>> > Sent from the Apache Ignite Developers mailing list >> archive at >> > >>>> > >>> Nabble.com. >> > >>>> > >>> > >> > >>>> > >>> >> > >>>> > >>> >> > >>>> > >>> >> > >>>> > >>> -- >> > >>>> > >>> Alexey Kuznetsov >> > >>>> > >>> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> -- >> > >>>> > >> Best Regards, Vyacheslav >> > >>>> > >> >> > >>>> > > >> > >>>> > > >> > >>>> > > >> > >>>> > > -- >> > >>>> > > Best Regards, Vyacheslav >> > >>>> > > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > -- >> > >>>> > Best Regards, Vyacheslav >> > >>>> > >> > >>>> >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> Best Regards, Vyacheslav >> > >>> >> > >> >> > >> >> > >> >> > >> -- >> > >> Best Regards, Vyacheslav >> > >> >> > > >> > > >> > > >> > > -- >> > > Best Regards, Vyacheslav >> > > >> > >> > >> > >> > -- >> > Best Regards, Vyacheslav >> > >> > > > > -- > Best Regards, Vyacheslav > -- Best Regards, Vyacheslav
