Hi, I've filed a ticket [1]. I'll try to share design details in a couple of days.
1. https://issues.apache.org/jira/browse/IGNITE-7024 2017-11-23 18:31 GMT+03:00 Denis Magda <dma...@apache.org>: > Nikita, > > Sounds like a good plan. Please share the design details prior getting > down to the implementation. > > — > Denis > > > On Nov 23, 2017, at 4:38 AM, Nikita Amelchev <nsamelc...@gmail.com> > wrote: > > > > Hi Igniters! > > > > I’m working on the similar feature for my own project. > > I would like to suggest use in-line compression and write encoded bytes > in > > network channel by bytes array buffer. It allows us avoiding expensive > > memory allocation. > > The described design may be implemented in TcpCommunicationSpi level. We > > can introduce pluggable compressor on TCP level where we will be able to > > describe our compression strategy, for example, exclude some small > messages > > and many other. > > > > If the community doesn't mind I will file the ticket and will start > > implementing it. > > Any thoughts? > > > > 2017-11-23 12:06 GMT+03:00 Vladimir Ozerov <voze...@gridgain.com>: > > > >> Denis, > >> > >> Regarding zipped marshaller - this would be inefficient, because > >> compression rate will be lower. > >> > >> On Thu, Nov 23, 2017 at 1:01 AM, Denis Magda <dma...@apache.org> wrote: > >> > >>> Nikita, > >>> > >>> Your solution sounds reasonable from the first glance. However, the > >>> communication layer processes a dozen of small system messages that > >> should > >>> be excluded from the compression. Guess, that we will spend more time > on > >>> compressing/decompressing them thus diminishing the positive effect of > >> the > >>> compression. > >>> > >>> Alexey K., Vladimir O., > >>> > >>> What if we create Zip version of the binary marshaller the same way we > >>> implemented GridClientZipOptimizedMarshaller? > >>> > >>> — > >>> Denis > >>> > >>>> On Nov 22, 2017, at 5:36 AM, Alexey Kuznetsov <akuznet...@apache.org> > >>> wrote: > >>>> > >>>> I think it is very useful feature. > >>>> I also have experience when server nodes connected via fast network. > >>>> But client nodes via very slow network. > >>>> > >>>> I implemeted GridClientZipOptimizedMarshaller and that solved my > >> issue. > >>>> But this marshaller works only with old > >>>> and org.apache.ignite.internal.client.GridClient and has a lot of > >>>> limitations. > >>>> But compression was about 6-20x times. > >>>> > >>>> We need a solution for Ignite 2.x and client nodes. > >>>> > >>>> > >>>> On Wed, Nov 22, 2017 at 7:48 PM, Nikita Amelchev < > nsamelc...@gmail.com > >>> > >>>> wrote: > >>>> > >>>>> Hello, Igniters! > >>>>> > >>>>> I think it is a useful feature. I suggest to implement it to > >>> communication > >>>>> SPI like SSL encryption implemented. I have experience with this > >> feature > >>>>> and I can try to develop it. > >>>>> > >>>>> 2017-11-22 12:01 GMT+03:00 Alexey Kukushkin < > >> kukushkinale...@gmail.com > >>>> : > >>>>> > >>>>>> Forwarding to DEV list: Ignite developers, could you please share > >> your > >>>>>> thoughts on how hard it is to extend Ignite to compress data on the > >>>>>> network. > >>>>>> > >>>>>> On Wed, Nov 22, 2017 at 10:04 AM, Gordon Reid (Nine Mile) < > >>>>>> gordon.r...@ninemilefinancial.com> wrote: > >>>>>> > >>>>>>> Hi Igniters, > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> I see there is a lot of discussion in certain threads about > >>>>> compression. > >>>>>>> This seems to have diverged into conversations about object versus > >>>>> field > >>>>>>> compression, and even throwing encryption into the mix. For my use > >>>>> case, > >>>>>> I > >>>>>>> am not interested in compressing the cache stored in memory, I have > >>>>>> plenty > >>>>>>> of memory for my application. What I don’t have is a good network. > I > >>>>>> have a > >>>>>>> high latency, low bandwidth network between my C# ignite client and > >> my > >>>>>> Java > >>>>>>> ignite server. I only want to compress data when it is sent over > the > >>>>>>> network to remote nodes. It should be stored in the local memory > >>>>>>> uncompressed. How can we achive this? Can the TcpCommunicationSpi > >>>>> support > >>>>>>> compression? > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Gordon. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> This email and any attachments are proprietary & confidential and > >> are > >>>>>>> intended solely for the use of the individuals to whom it is > >>> addressed. > >>>>>> Any > >>>>>>> views or opinions expressed are solely for those of the author and > >> do > >>>>> not > >>>>>>> necessarily reflect those of Nine Mile Financial Pty. Limited. If > >> you > >>>>>> have > >>>>>>> received this email in error, please let us know immediately by > >> reply > >>>>>> email > >>>>>>> and delete from your system. Nine Mile Financial Pty. Limited. ABN: > >>> 346 > >>>>>>> 1349 0252 > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Best regards, > >>>>>> Alexey > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Best wishes, > >>>>> Amelchev Nikita > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Alexey Kuznetsov > >>> > >>> > >> > > > > > > > > -- > > Best wishes, > > Amelchev Nikita > > -- Best wishes, Amelchev Nikita
