Hi Igniters.
Vladimir, I want to propose another design of an implementation of the
per-field compression.
1) We will add new step in the method prepareForCache (for example) of
CacheObject, or in GridCacheMapEntry.
At the step, after marshalling of an object, we will compress fields of the
object which described in advance.
User will describe class fields which he wants to compess in an another
entity like Metadata.
For compression, we will introduce another entity, for example
CompressionProcessor, which will work with bytes array (marshalled object).
The entity will read bytes array of described fields, compress it and
rewrite binary representation of the whole object.
After processing the object will be put in the cache.
In this case design not to relate to binary infrastructure.
But there is big overhead to heap-memory for the buffer.
2) Another solution is to compress bytes array of whole object on copying
to off-heap.
But, in this case I don't understand yet, how to provide support of
querying and indexing.
2017-06-09 11:21 GMT+03:00 Sergey Kozlov <skoz...@gridgain.com>:
> Hi
>
> * "Per-field compression" is applicable for huge BLOB fields and will
> impose the restrictions like unable ot index such fields, slower getting
> data, potential OOM issues if compression ration is too high.
> But for some cases it makes sense
>
> On Fri, Jun 9, 2017 at 11:11 AM, Антон Чураев <churaev...@gmail.com>
> wrote:
>
> > Seems that Dmitry is referring to transparent data encryption. It is used
> > throughout the whale database industry.
> >
> > 2017-06-09 10:50 GMT+03:00 Vladimir Ozerov <voze...@gridgain.com>:
> >
> > > Dima,
> > >
> > > Encryption of certain fields is as bad as compression. First, it is a
> > huge
> > > change, which makes already complex binary protocol even more complex.
> > > Second, it have to be ported to CPP, .NET platforms, as well as to JDBC
> > and
> > > ODBC.
> > > Last, but the most important - this is not our headache to encrypt
> > > sensitive data. This is user responsibility. Nobody in a sane mind will
> > > store passwords in plain form. Instead, user should encrypt it on his
> > own,
> > > choosing proper encryption parameters - algorithms, key lengths, salts,
> > > etc.. How are you going to expose this in API or configuration?
> > >
> > > We should not implement data encryption on binary level, this is out of
> > > question. Encryption should be implemented on application level (user
> > > efforts), transport layer (SSL - we already have it), and possibly on
> > > disk-level (there are tools for this already).
> > >
> > >
> > > On Fri, Jun 9, 2017 at 9:06 AM, Vyacheslav Daradur <
> daradu...@gmail.com>
> > > wrote:
> > >
> > > > >> which is much less useful.
> > > > I note, in some cases there is profit more than twice per size of an
> > > > object.
> > > >
> > > > >> Would it be possible to change your implementation to handle the
> > > > encryption instead?
> > > > Yes, of cource, there's not much difference between compression and
> > > > encryption, including in my implementation of per-field-compression.
> > > >
> > > > 2017-06-09 8:55 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>:
> > > >
> > > > > Vyacheslav,
> > > > >
> > > > > When this feature started out as data compression in Ignite, it
> > sounded
> > > > > very useful. Now it is unfolding as a per-field compression, which
> is
> > > > much
> > > > > less useful. In fact, it is questionable whether it is useful at
> all.
> > > The
> > > > > fact that this feature is implemented does not make it mandatory
> for
> > > the
> > > > > community to accept it.
> > > > >
> > > > > However, as I mentioned before, per-field encryption is very
> useful,
> > as
> > > > it
> > > > > would allow users automatically encrypt certain sensitive fields,
> > like
> > > > > passwords, credit card numbers, etc. There is not much conceptual
> > > > > difference between compressing a field vs encrypting a field. Would
> > it
> > > be
> > > > > possible to change your implementation to handle the encryption
> > > instead?
> > > > >
> > > > > D.
> > > > >
> > > > > On Thu, Jun 8, 2017 at 10:42 PM, Vyacheslav Daradur <
> > > daradu...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Guys, I want to be clear:
> > > > > > * "Per-field compression" design is the result of a research of
> the
> > > > > binary
> > > > > > infrastructure of Ignite and some other its places (querying,
> > > indexing,
> > > > > > etc.)
> > > > > > * Full-compression of object will be more effective, but in this
> > case
> > > > > there
> > > > > > is no capability with querying and indexing (or there is large
> > > overhead
> > > > > by
> > > > > > way of decompressing of full object (or caches pages) on demand)
> > > > > > * "Per-field compression" is a one of ways to implement the
> > > compression
> > > > > > feature
> > > > > >
> > > > > > I'm new to Ignite also I can be mistaken in some things.
> > > > > > Last 3-4 month I've tryed to start dicussion about a design, but
> > > nobody
> > > > > > answers nothing (except Dmitry and Valentin who was interested
> how
> > it
> > > > > > works).
> > > > > > But I understand that this is community and nobody is obliged to
> > > > anybody.
> > > > > >
> > > > > > There are strong Ignite experts.
> > > > > > If they can help me and community with a design of the
> compression
> > > > > feature
> > > > > > it will be great.
> > > > > > At the moment I have a desire and time to be engaged in
> development
> > > of
> > > > > > compression feature in Ignite.
> > > > > > Let's use this opportunity :)
> > > > > >
> > > > > > 2017-06-09 5:36 GMT+03:00 Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I have never seen a single Ignite user asking about
> compressing a
> > > > > single
> > > > > > > field. However, we have had requests to secure certain fields,
> > e.g.
> > > > > > > passwords.
> > > > > > >
> > > > > > > I personally do not think per-field compression is needed,
> unless
> > > we
> > > > > can
> > > > > > > point out some concrete real life use cases.
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Thu, Jun 8, 2017 at 3:42 AM, Vyacheslav Daradur <
> > > > > daradu...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Anton,
> > > > > > > >
> > > > > > > > >> I thought that if there will storing compressed data in
> the
> > > > > memory,
> > > > > > > data
> > > > > > > > >> will transmit over wire in compression too. Is it right?
> > > > > > > >
> > > > > > > > In per-field compression case - yes.
> > > > > > > >
> > > > > > > > 2017-06-08 13:36 GMT+03:00 Антон Чураев <
> churaev...@gmail.com
> > >:
> > > > > > > >
> > > > > > > > > Guys, could you please help me.
> > > > > > > > > I thought that if there will storing compressed data in the
> > > > memory,
> > > > > > > data
> > > > > > > > > will transmit over wire in compression too. Is it right?
> > > > > > > > >
> > > > > > > > > 2017-06-08 13:30 GMT+03:00 Vyacheslav Daradur <
> > > > daradu...@gmail.com
> > > > > >:
> > > > > > > > >
> > > > > > > > > > Vladimir,
> > > > > > > > > >
> > > > > > > > > > The main problem which I'am trying to solve is storing
> data
> > > in
> > > > > > memory
> > > > > > > > in
> > > > > > > > > a
> > > > > > > > > > compression form via Ignite.
> > > > > > > > > > The main goal is using memory more effectivelly.
> > > > > > > > > >
> > > > > > > > > > >> here the much simpler step would be to full
> > > > > > > > > > compression on per-cache basis rather than dealing with
> > > > > per-fields
> > > > > > > > case.
> > > > > > > > > >
> > > > > > > > > > Please explain your idea. Compess data by memory-page?
> > > > > > > > > > Is it compatible with quering and indexing?
> > > > > > > > > >
> > > > > > > > > > >> In the end, if user would like to compress particular
> > > field,
> > > > > he
> > > > > > > can
> > > > > > > > > > always to it on his own
> > > > > > > > > > I think we mustn't think in this way, if user need
> > something
> > > he
> > > > > > > trying
> > > > > > > > to
> > > > > > > > > > choose a tool which has this feature OOTB.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 2017-06-08 12:53 GMT+03:00 Vladimir Ozerov <
> > > > voze...@gridgain.com
> > > > > >:
> > > > > > > > > >
> > > > > > > > > > > Igniters,
> > > > > > > > > > >
> > > > > > > > > > > Honestly I still do not see how to apply it gracefully
> > this
> > > > > > feature
> > > > > > > > ti
> > > > > > > > > > > Ignite. And overall approach to compress only
> particular
> > > > fields
> > > > > > > looks
> > > > > > > > > > > overcomplicated to me. Remember, that our main use case
> > is
> > > an
> > > > > > > > > application
> > > > > > > > > > > without classes on the server. It means that any kind
> of
> > > > > > > annotations
> > > > > > > > > are
> > > > > > > > > > > inapplicable. To be more precise: proper API should be
> > > > > > implemented
> > > > > > > to
> > > > > > > > > > > handle no-class case (e.g. how would build such an
> object
> > > > > through
> > > > > > > > > > > BinaryBuilder without a class?), and only then add
> > > > annotations
> > > > > as
> > > > > > > > > > > convenient addition to more basic API.
> > > > > > > > > > >
> > > > > > > > > > > It seems to me that full implementation, which takes in
> > > count
> > > > > > > proper
> > > > > > > > > > > "classless" API, changes to binary metadata to reflect
> > > > > compressed
> > > > > > > > > fields,
> > > > > > > > > > > changes to SQL, changes to binary protocol, and porting
> > to
> > > > .NET
> > > > > > and
> > > > > > > > > CPP,
> > > > > > > > > > > will yield very complex solution with little value to
> the
> > > > > > product.
> > > > > > > > > > >
> > > > > > > > > > > Instead, as I proposed earlier, it seems that we'd
> better
> > > > start
> > > > > > > with
> > > > > > > > > the
> > > > > > > > > > > problem we are trying to solve. Basically, compression
> > > could
> > > > > help
> > > > > > > in
> > > > > > > > > two
> > > > > > > > > > > cases:
> > > > > > > > > > > 1) Transmitting data over wire - it should be
> implemented
> > > on
> > > > > > > > > > communication
> > > > > > > > > > > layer and should not affect binary serialization
> > component
> > > a
> > > > > lot.
> > > > > > > > > > > 2) Storing data in memory - here the much simpler step
> > > would
> > > > be
> > > > > > to
> > > > > > > > full
> > > > > > > > > > > compression on per-cache basis rather than dealing with
> > > > > > per-fields
> > > > > > > > > case.
> > > > > > > > > > >
> > > > > > > > > > > In the end, if user would like to compress particular
> > > field,
> > > > he
> > > > > > can
> > > > > > > > > > always
> > > > > > > > > > > to it on his own, and set already compressed field to
> our
> > > > > > > > BinaryObject.
> > > > > > > > > > >
> > > > > > > > > > > Vladimir.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Jun 8, 2017 at 12:37 PM, Vyacheslav Daradur <
> > > > > > > > > daradu...@gmail.com
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Valentin,
> > > > > > > > > > > >
> > > > > > > > > > > > Yes, I have the prototype[1][2]
> > > > > > > > > > > >
> > > > > > > > > > > > You can see an example of Java class[3] that I used
> in
> > my
> > > > > > > > benchmark.
> > > > > > > > > > > > For example:
> > > > > > > > > > > > class Foo {
> > > > > > > > > > > > @BinaryCompression
> > > > > > > > > > > > String data;
> > > > > > > > > > > > }
> > > > > > > > > > > > If user make decision to store the object in
> compressed
> > > > form,
> > > > > > he
> > > > > > > > can
> > > > > > > > > > use
> > > > > > > > > > > > the annotation @BinaryCompression as shown above.
> > > > > > > > > > > > It means annotated field 'data' will be compressed at
> > > > > > > marshalling.
> > > > > > > > > > > >
> > > > > > > > > > > > [1] https://github.com/apache/ignite/pull/1951
> > > > > > > > > > > > [2] https://issues.apache.org/
> jira/browse/IGNITE-5226
> > > > > > > > > > > > [3]
> > > > > > > > > > > > https://github.com/daradurvs/
> ignite-compression/blob/
> > > > > > > > > > > > master/src/main/java/ru/
> daradurvs/ignite/compression/
> > > > > > > > > > model/Audit1F.java
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > 2017-06-08 2:04 GMT+03:00 Valentin Kulichenko <
> > > > > > > > > > > > valentin.kuliche...@gmail.com
> > > > > > > > > > > > >:
> > > > > > > > > > > >
> > > > > > > > > > > > > Vyacheslav, Anton,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Are there any ideas and/or prototypes for the API?
> > Your
> > > > > > design
> > > > > > > > > > > > suggestions
> > > > > > > > > > > > > seem to make sense, but I would like to see how it
> > all
> > > > this
> > > > > > > will
> > > > > > > > > like
> > > > > > > > > > > > from
> > > > > > > > > > > > > user's standpoint.
> > > > > > > > > > > > >
> > > > > > > > > > > > > -Val
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, Jun 7, 2017 at 1:06 AM, Антон Чураев <
> > > > > > > > churaev...@gmail.com
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Vyacheslav, correct me if something wrong
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > We could provide opportunity of choose between
> CPU
> > > > usage
> > > > > > and
> > > > > > > > > > MEM/NET
> > > > > > > > > > > > > usage
> > > > > > > > > > > > > > for users by compression some attributes of
> stored
> > > > > objects.
> > > > > > > > > > > > > > You have learned design, and it is possible to
> > > localize
> > > > > > > changes
> > > > > > > > > in
> > > > > > > > > > > > > > marshalling without performance affect and
> current
> > > > > > > > functionality.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I think, that it's usefull for our project and
> > users.
> > > > > > > > > > > > > > Community, what do you think about this proposal?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2017-06-06 17:29 GMT+03:00 Vyacheslav Daradur <
> > > > > > > > > daradu...@gmail.com
> > > > > > > > > > >:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > In short,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > During marshalling a fields is represented as
> > > > > > > > > BinaryFieldAccessor
> > > > > > > > > > > > which
> > > > > > > > > > > > > > > manages its marshalling. It checks if the field
> > is
> > > > > marked
> > > > > > > by
> > > > > > > > > > > > annotation
> > > > > > > > > > > > > > > @BinaryCompression, in that case - binary
> > > > > representation
> > > > > > > of
> > > > > > > > > > field
> > > > > > > > > > > > > (bytes
> > > > > > > > > > > > > > > array) will be compressed. It will be marked as
> > > > > > compressed
> > > > > > > by
> > > > > > > > > > types
> > > > > > > > > > > > > > > constant (GridBinaryMarshaller.COMPRESSED),
> > after
> > > > this
> > > > > > the
> > > > > > > > > > > > compressed
> > > > > > > > > > > > > > > bytes
> > > > > > > > > > > > > > > array wiil be include in binary representation
> of
> > > > whole
> > > > > > > > object.
> > > > > > > > > > > Note,
> > > > > > > > > > > > > > > header of marshalled object will not be
> > compressed.
> > > > > > > > Compression
> > > > > > > > > > > > > affected
> > > > > > > > > > > > > > > only object's field representation.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Objects in IgniteCache is represented as
> > > BinaryObject
> > > > > > which
> > > > > > > > is
> > > > > > > > > > > > wrapper
> > > > > > > > > > > > > > over
> > > > > > > > > > > > > > > bytes array of marshalled object.
> > > > > > > > > > > > > > > BinaryObject provides some usefull methods,
> which
> > > are
> > > > > > used
> > > > > > > by
> > > > > > > > > > > Ignite
> > > > > > > > > > > > > > > systems.
> > > > > > > > > > > > > > > For example, the Queries use BinaryObject#field
> > > > method,
> > > > > > > which
> > > > > > > > > > > > > > deserializes
> > > > > > > > > > > > > > > only field of object, without deserializing of
> > > whole
> > > > > > > object.
> > > > > > > > > > > > > > > BinaryObject#field method during
> deserialization,
> > > if
> > > > > > meets
> > > > > > > > the
> > > > > > > > > > > > constant
> > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > compressed type, decompress this bytes array,
> > then
> > > > > > continue
> > > > > > > > > > > > > unmarshalling
> > > > > > > > > > > > > > > as usual.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Now, I introduced the Compressor interface in
> > > > > > > > > > IgniteConfigurations,
> > > > > > > > > > > > it
> > > > > > > > > > > > > > > allows user to use own implementation of
> > > compressor -
> > > > > it
> > > > > > is
> > > > > > > > the
> > > > > > > > > > > > > > requirement
> > > > > > > > > > > > > > > in the task[1].
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > As far as I know, Vladimir Ozerov doesn't like
> > the
> > > > idea
> > > > > > of
> > > > > > > > > > granting
> > > > > > > > > > > > > this
> > > > > > > > > > > > > > > opportunity to the user.
> > > > > > > > > > > > > > > In that case we can choose a compression
> > algorithm
> > > > > which
> > > > > > we
> > > > > > > > > will
> > > > > > > > > > > > > provide
> > > > > > > > > > > > > > by
> > > > > > > > > > > > > > > default and will move the interface to
> internals
> > of
> > > > > > binary
> > > > > > > > > > > > > > infractructure.
> > > > > > > > > > > > > > > For this case I've prepared benchmarked, which
> > I've
> > > > > sent
> > > > > > > > > earlier.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I vote for ZSTD algorithm[2], it provides good
> > > > > > compression
> > > > > > > > > ratio
> > > > > > > > > > > and
> > > > > > > > > > > > > good
> > > > > > > > > > > > > > > throughput. It has implementation in Java, .NET
> > and
> > > > > C++,
> > > > > > > and
> > > > > > > > > has
> > > > > > > > > > > > > > > ASF-friendly license, we can use it in the all
> > > Ignite
> > > > > > > > > platforms.
> > > > > > > > > > > > > > > You can look at an assessment of this algorithm
> > in
> > > my
> > > > > > > > > benchmark's
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > [1] https://issues.apache.org/
> > > > jira/browse/IGNITE-3592
> > > > > > > > > > > > > > > [2]https://github.com/facebook/zstd
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 2017-06-06 16:02 GMT+03:00 Антон Чураев <
> > > > > > > > churaev...@gmail.com
> > > > > > > > > >:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Looks good for me.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Could You propose design of implementation in
> > > > couple
> > > > > of
> > > > > > > > > > > sentences?
> > > > > > > > > > > > > > > > So that we can estimate the completeness and
> > > > > complexity
> > > > > > > of
> > > > > > > > > the
> > > > > > > > > > > > > > proposal.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 2017-06-06 15:26 GMT+03:00 Vyacheslav
> Daradur <
> > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > >:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Anton,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Of course, the solution does not affect on
> > > > existing
> > > > > > > > > > > > > implementation. I
> > > > > > > > > > > > > > > > mean,
> > > > > > > > > > > > > > > > > there is no changes if user not use the
> > > > annotation
> > > > > > > > > > > > > > @BinaryCompression.
> > > > > > > > > > > > > > > > (no
> > > > > > > > > > > > > > > > > performance changes)
> > > > > > > > > > > > > > > > > Only if user make decision to use
> compression
> > > on
> > > > > > > specific
> > > > > > > > > > field
> > > > > > > > > > > > or
> > > > > > > > > > > > > > > fields
> > > > > > > > > > > > > > > > > of a class - in that case compression will
> be
> > > > used
> > > > > at
> > > > > > > > > > > marshalling
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > relation to annotated fields.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > 2017-06-06 15:10 GMT+03:00 Антон Чураев <
> > > > > > > > > > churaev...@gmail.com
> > > > > > > > > > > >:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Vyacheslav,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Is it possible to propose implementation
> > that
> > > > can
> > > > > > be
> > > > > > > > > > switched
> > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > on-demand?
> > > > > > > > > > > > > > > > > > In this case it should not affect
> > performance
> > > > of
> > > > > > > > current
> > > > > > > > > > > > > solution.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I mean, that users should make decision
> > what
> > > is
> > > > > > more
> > > > > > > > > > > important
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > them:
> > > > > > > > > > > > > > > > > > throutput or memory/net usage.
> > > > > > > > > > > > > > > > > > May be they will be choose not all
> objects,
> > > or
> > > > > only
> > > > > > > > some
> > > > > > > > > > > > > attributes
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > objects for compress.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 2017-06-06 14:48 GMT+03:00 Vyacheslav
> > > Daradur <
> > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Conclusion:
> > > > > > > > > > > > > > > > > > > Provided solution allows reduce size of
> > an
> > > > > object
> > > > > > > in
> > > > > > > > > > > > > IgniteCache
> > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > cost of throughput reduction (small -
> in
> > > some
> > > > > > > cases),
> > > > > > > > > it
> > > > > > > > > > > > > depends
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > part
> > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > object which will be compressed and
> > > > compression
> > > > > > > > > > algorithm.
> > > > > > > > > > > > > > > > > > > I mean, we can make more effective use
> of
> > > > > memory,
> > > > > > > and
> > > > > > > > > in
> > > > > > > > > > > some
> > > > > > > > > > > > > > cases
> > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > > > reduce loading of the interconnect.
> > > > > (replication,
> > > > > > > > > > > > rebalancing)
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Especially, it will be particularly
> > useful
> > > > for
> > > > > > > > object's
> > > > > > > > > > > > fields
> > > > > > > > > > > > > > > which
> > > > > > > > > > > > > > > > > are
> > > > > > > > > > > > > > > > > > > large text (>~ 250 bytes) and can be
> > > > > effectively
> > > > > > > > > > > compressed.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > 2017-06-06 12:00 GMT+03:00 Антон
> Чураев <
> > > > > > > > > > > > churaev...@gmail.com
> > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Vyacheslav, thank you! But could you
> > > please
> > > > > > > > provide a
> > > > > > > > > > > > > > conclusions
> > > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > > > > proposals based on this benchmarks?
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > 2017-06-06 11:28 GMT+03:00 Vyacheslav
> > > > > Daradur <
> > > > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Dmitry,
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Excel-pages:
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > 1). "Compression ratio (2)" - shows
> > > > object
> > > > > > > size,
> > > > > > > > > with
> > > > > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > > > without compression. (Conditions:
> > > literal
> > > > > > text)
> > > > > > > > > > > > > > > > > > > > > 1st graph shows compression ratios
> of
> > > > using
> > > > > > > > > different
> > > > > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > > > > algrithms
> > > > > > > > > > > > > > > > > > > > > depending on size of compressed
> > field.
> > > > > > > > > > > > > > > > > > > > > 2nd graph shows evaluation of size
> of
> > > > > objects
> > > > > > > > > > depending
> > > > > > > > > > > > on
> > > > > > > > > > > > > > > sizes
> > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > > > compression algorithms.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > 2). "Compression ratio (1)" - shows
> > > > object
> > > > > > > size,
> > > > > > > > > with
> > > > > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > > > without compression. (Conditions:
> > > badly
> > > > > > > > compressed
> > > > > > > > > > > > > character
> > > > > > > > > > > > > > > > > > sequence)
> > > > > > > > > > > > > > > > > > > > > 1st graph shows compression ratios
> of
> > > > using
> > > > > > > > > different
> > > > > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > > > > > algrithms depending on size of
> > > compressed
> > > > > > > field.
> > > > > > > > > > > > > > > > > > > > > 2nd graph shows evaluation of size
> of
> > > > > objects
> > > > > > > > > > depending
> > > > > > > > > > > > on
> > > > > > > > > > > > > > > sizes
> > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > > > compression algorithms.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > 3) 'put-avg" - shows average time
> of
> > > the
> > > > > > "put"
> > > > > > > > > > > operation
> > > > > > > > > > > > > > > > depending
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > size
> > > > > > > > > > > > > > > > > > > > > and compression algorithms.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > 4) 'put-thrpt" - shows throughput
> of
> > > the
> > > > > > "put"
> > > > > > > > > > > operation
> > > > > > > > > > > > > > > > depending
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > size
> > > > > > > > > > > > > > > > > > > > > and compression algorithms.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > 5) 'get-avg" - shows average time
> of
> > > the
> > > > > > "get"
> > > > > > > > > > > operation
> > > > > > > > > > > > > > > > depending
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > size
> > > > > > > > > > > > > > > > > > > > > and compression algorithms.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > 6) 'get-thrpt" - shows throughput
> of
> > > the
> > > > > > "get"
> > > > > > > > > > > operation
> > > > > > > > > > > > > > > > depending
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > > size
> > > > > > > > > > > > > > > > > > > > > and compression algorithms.
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > 2017-06-06 10:59 GMT+03:00 Dmitriy
> > > > > Setrakyan
> > > > > > <
> > > > > > > > > > > > > > > > > dsetrak...@apache.org
> > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > Vladimir, I am not sure how to
> > > > interpret
> > > > > > the
> > > > > > > > > > graphs?
> > > > > > > > > > > > What
> > > > > > > > > > > > > > are
> > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > > looking
> > > > > > > > > > > > > > > > > > > > > > at?
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 6, 2017 at 12:33 AM,
> > > > > Vyacheslav
> > > > > > > > > > Daradur <
> > > > > > > > > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > Hi, Igniters.
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > I've prepared some
> benchmarking.
> > > > > Results
> > > > > > > [1].
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > And I've prepared the
> evaluation
> > in
> > > > the
> > > > > > > form
> > > > > > > > of
> > > > > > > > > > > > > diagrams
> > > > > > > > > > > > > > > [2].
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > I hope that helps to interest
> the
> > > > > > community
> > > > > > > > and
> > > > > > > > > > > > > > > accelerates a
> > > > > > > > > > > > > > > > > > > > reaction
> > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > this improvment :)
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > [1]
> > > > > > > > > > > > > > > > > > > > > > > https://github.com/daradurvs/
> > > > > > > > > > > > ignite-compression/tree/
> > > > > > > > > > > > > > > > > > > > > > > master/src/main/resources/
> result
> > > > > > > > > > > > > > > > > > > > > > > [2]
> > https://drive.google.com/file/
> > > d/
> > > > > > > > > > > > > > > > > > 0B2CeUAOgrHkoMklyZ25YTEdKcEk/
> > > > > > > > > > > > > > > > > > > > view
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > 2017-05-24 9:49 GMT+03:00
> > > Vyacheslav
> > > > > > > Daradur
> > > > > > > > <
> > > > > > > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > Guys, any thoughts?
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > 2017-05-16 13:40 GMT+03:00
> > > > Vyacheslav
> > > > > > > > > Daradur <
> > > > > > > > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >> Hi guys,
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >> I've prepared the PR to show
> > my
> > > > > idea.
> > > > > > > > > > > > > > > > > > > > > > > >> https://github.com/apache/
> > > > > > > > > > > ignite/pull/1951/files
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >> About querying - I've just
> > > copied
> > > > > > > existing
> > > > > > > > > > tests
> > > > > > > > > > > > and
> > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > > > > annotated
> > > > > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > > > >> testing data.
> > > > > > > > > > > > > > > > > > > > > > > >> https://github.com/apache/
> > > > > > > > > > > > > > ignite/pull/1951/files#diff-
> > > > > > > > > > > > > > > > > c19a9d
> > > > > > > > > > > > > > > > > > > > > > > >> f4058141d059bb577e75244764
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >> It means fields which will
> be
> > > > marked
> > > > > > by
> > > > > > > > > > > > > > > @BinaryCompression
> > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > > > > > >> compressed at marshalling
> via
> > > > > > > > > > BinaryMarshaller.
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >> This solution has no effect
> on
> > > > > > existing
> > > > > > > > data
> > > > > > > > > > or
> > > > > > > > > > > > > > project
> > > > > > > > > > > > > > > > > > > > > architecture.
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >> I'll be glad to see your
> > > thougths.
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >> 2017-05-15 19:18 GMT+03:00
> > > > > Vyacheslav
> > > > > > > > > Daradur
> > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >>> Dmitriy,
> > > > > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>> I have ready prototype. I
> > want
> > > to
> > > > > > show
> > > > > > > > it.
> > > > > > > > > > > > > > > > > > > > > > > >>> It is always easier to
> > discuss
> > > on
> > > > > > > > example.
> > > > > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>> 2017-05-15 19:02 GMT+03:00
> > > > Dmitriy
> > > > > > > > > Setrakyan
> > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > > dsetrak...@apache.org
> > > > > > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> Vyacheslav,
> > > > > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> I think it is a bit
> > premature
> > > to
> > > > > > > > provide a
> > > > > > > > > > PR
> > > > > > > > > > > > > > without
> > > > > > > > > > > > > > > > > > getting
> > > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > > > > >>>> community
> > > > > > > > > > > > > > > > > > > > > > > >>>> consensus on the dev list.
> > > > Please
> > > > > > > allow
> > > > > > > > > some
> > > > > > > > > > > > time
> > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > > community
> > > > > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > >>>> respond.
> > > > > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> D.
> > > > > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> On Mon, May 15, 2017 at
> 6:36
> > > AM,
> > > > > > > > > Vyacheslav
> > > > > > > > > > > > > Daradur
> > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > > > > >>>> daradu...@gmail.com>
> > > > > > > > > > > > > > > > > > > > > > > >>>> wrote:
> > > > > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > I created the ticket:
> > > > > > > > > > > > > > > https://issues.apache.org/jira
> > > > > > > > > > > > > > > > > > > > > > > >>>> /browse/IGNITE-5226
> > > > > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > I'll prepare a PR with
> > > > described
> > > > > > > > > solution
> > > > > > > > > > in
> > > > > > > > > > > > > > couple
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > days.
> > > > > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > 2017-05-15 15:05
> GMT+03:00
> > > > > > > Vyacheslav
> > > > > > > > > > > Daradur
> > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > Hi, Igniters!
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > Apache 2.0 is
> released.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > Let's continue the
> > > > discussion
> > > > > > > about
> > > > > > > > a
> > > > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > > design.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > At the moment, I found
> > > only
> > > > > one
> > > > > > > > > solution
> > > > > > > > > > > > which
> > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > > compatible
> > > > > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > > > > >>>> > querying
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > and indexing, this is
> > > > > > > > > per-objects-field
> > > > > > > > > > > > > > > compression.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > Per-fields compression
> > > means
> > > > > > that
> > > > > > > > > > metadata
> > > > > > > > > > > > (a
> > > > > > > > > > > > > > > > header)
> > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > an
> > > > > > > > > > > > > > > > > > > > > > object
> > > > > > > > > > > > > > > > > > > > > > > >>>> won't
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > be compressed, only
> > > > serialized
> > > > > > > > values
> > > > > > > > > of
> > > > > > > > > > > an
> > > > > > > > > > > > > > object
> > > > > > > > > > > > > > > > > > fields
> > > > > > > > > > > > > > > > > > > > (in
> > > > > > > > > > > > > > > > > > > > > > > bytes
> > > > > > > > > > > > > > > > > > > > > > > >>>> array
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > form) will be
> > compressed.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > This solution have
> some
> > > > > > > contentious
> > > > > > > > > > > issues:
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > - small values, like
> > > > > primitives
> > > > > > > and
> > > > > > > > > > short
> > > > > > > > > > > > > > arrays -
> > > > > > > > > > > > > > > > > there
> > > > > > > > > > > > > > > > > > > > isn't
> > > > > > > > > > > > > > > > > > > > > > > >>>> sense to
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > compress them;
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > - there is no possible
> > to
> > > > use
> > > > > > > > > > compression
> > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > java-predefined
> > > > > > > > > > > > > > > > > > > > > > > >>>> types;
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > We can provide an
> > > > annotation,
> > > > > > > > > > > > > > @IgniteCompression -
> > > > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > > > > > example,
> > > > > > > > > > > > > > > > > > > > > > > >>>> which can
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > be used by users for
> > > marking
> > > > > > > fields
> > > > > > > > to
> > > > > > > > > > > > > compress.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > Any thoughts?
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > Maybe someone already
> > have
> > > > > ready
> > > > > > > > > design?
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > 2017-04-10 11:06
> > GMT+03:00
> > > > > > > > Vyacheslav
> > > > > > > > > > > > Daradur
> > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > > > > > > > > > > >>>> >:
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> Alexey,
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> Yes, I've read it.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> Ok, let's discuss
> about
> > > > > public
> > > > > > > API
> > > > > > > > > > > design.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> I think we need to
> add
> > > > some a
> > > > > > > > > configure
> > > > > > > > > > > > > entity
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > > > >>>> CacheConfiguration,
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> which will contain
> the
> > > > > > Compressor
> > > > > > > > > > > interface
> > > > > > > > > > > > > > > > > > > implementation
> > > > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > > > > >>>> > usefull
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> parameters.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> Or maybe to provide a
> > > > > > > > > BinaryMarshaller
> > > > > > > > > > > > > > decorator,
> > > > > > > > > > > > > > > > > which
> > > > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > > > > > >>>> compress
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> data after
> marshalling.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> 2017-04-10 10:40
> > > GMT+03:00
> > > > > > Alexey
> > > > > > > > > > > > Kuznetsov <
> > > > > > > > > > > > > > > > > > > > > > > akuznet...@apache.org
> > > > > > > > > > > > > > > > > > > > > > > >>>> >:
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> Vyacheslav,
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> Did you read initial
> > > > > > discussion
> > > > > > > > [1]
> > > > > > > > > > > about
> > > > > > > > > > > > > > > > > compression?
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> As far as I remember
> > we
> > > > > agreed
> > > > > > > to
> > > > > > > > > add
> > > > > > > > > > > only
> > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > "top-level"
> > > > > > > > > > > > > > > > > > > > > > API
> > > > > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > > > > >>>> > order
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> to
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> provide a way for
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> Ignite users to
> inject
> > > > some
> > > > > > sort
> > > > > > > > of
> > > > > > > > > > > custom
> > > > > > > > > > > > > > > > > > compression.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> [1]
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > http://apache-ignite-developer
> > > > > > > > > > > > > > > s.2346864.n4.nabble
> > > > > > > > > > > > > > > > .
> > > > > > > > > > > > > > > > > > > > > com/Data-c
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > ompression-in-Ignite-2-0-
> > > > > > > > > td10099.html
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> On Mon, Apr 10, 2017
> > at
> > > > 2:19
> > > > > > PM,
> > > > > > > > > > > > daradurvs <
> > > > > > > > > > > > > > > > > > > > > > daradu...@gmail.com
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > wrote:
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > Hi Igniters!
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > I am interested in
> > > this
> > > > > > task.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > Provide some kind
> of
> > > > > > pluggable
> > > > > > > > > > > > compression
> > > > > > > > > > > > > > SPI
> > > > > > > > > > > > > > > > > > support
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > <
> > > > > https://issues.apache.org/
> > > > > > > > > > > > > > > > > jira/browse/IGNITE-3592>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > I developed a
> > solution
> > > > on
> > > > > > > > > > > > > > > > BinaryMarshaller-level,
> > > > > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > > > > > > reviewer
> > > > > > > > > > > > > > > > > > > > > > > >>>> has
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> rejected
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > it.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > Let's continue
> > > > discussion
> > > > > of
> > > > > > > > task
> > > > > > > > > > > goals
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > solution
> > > > > > > > > > > > > > > > > > > > > design.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > As I understood
> > that,
> > > > the
> > > > > > main
> > > > > > > > > goal
> > > > > > > > > > of
> > > > > > > > > > > > > this
> > > > > > > > > > > > > > > task
> > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > > store
> > > > > > > > > > > > > > > > > > > > > > > >>>> data in
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > compressed form.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > This is what I
> need
> > > from
> > > > > > > Ignite
> > > > > > > > as
> > > > > > > > > > its
> > > > > > > > > > > > > user.
> > > > > > > > > > > > > > > > > > > Compression
> > > > > > > > > > > > > > > > > > > > > > > >>>> provides
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> economy
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > on
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > servers.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > We can store more
> > data
> > > > on
> > > > > > same
> > > > > > > > > > servers
> > > > > > > > > > > > at
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > cost
> > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > > >>>> increasing CPU
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > utilization.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > I'm researching a
> > > > > > possibility
> > > > > > > of
> > > > > > > > > > > > > > > implementation
> > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > > > > > > > >>>> at the
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > cache-level.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > Any thoughts?
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > --
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > Best regards,
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > Vyacheslav
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > --
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > View this message
> in
> > > > > > context:
> > > > > > > > > > > > > > > > > http://apache-ignite-
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > developers.2346864.n4.nabble.
> > > > > > > > > > > > > > > > > > com/Data-compression-in-
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > Ignite-2-0-tp10099p16317.html
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> > Sent from the
> Apache
> > > > > Ignite
> > > > > > > > > > Developers
> > > > > > > > > > > > > > mailing
> > > > > > > > > > > > > > > > > list
> > > > > > > > > > > > > > > > > > > > > archive
> > > > > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> Nabble.com.
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> --
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>> Alexey Kuznetsov
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> --
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >> Best Regards,
> > Vyacheslav
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > --
> > > > > > > > > > > > > > > > > > > > > > > >>>> > > Best Regards,
> Vyacheslav
> > > > > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>> > --
> > > > > > > > > > > > > > > > > > > > > > > >>>> > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>> --
> > > > > > > > > > > > > > > > > > > > > > > >>> Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >> --
> > > > > > > > > > > > > > > > > > > > > > > >> Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best Regards, Vyacheslav
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best Regards, Vyacheslav
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards, Vyacheslav
> > > >
> > >
> >
> >
> >
> > --
> >
> > Best Regards, Anton Churaev
> >
>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>
--
Best Regards, Vyacheslav
