Hi, Unfortunately, a quick fix did not give us too much performance boost.
I'm going to implement a complete algorithm change for storing the page identifier. But this change is quite significant and will require re-testing. I suggest including https://issues.apache.org/jira/browse/IGNITE-7638 in the next version, for example, to 2.5. Sincerely, Dmitriy Pavlov ср, 7 февр. 2018 г. в 22:27, Dmitriy Setrakyan <dsetrak...@apache.org>: > Agree on both, the performance fix and the spark data frames. Let's get > them into the release. > > However, Raymond is right. We should know how long the performance fix will > take. If it adds another month to the development, we should include it > into the next release. I am hoping that it can be done faster though. > > > Alexey Goncharuk, Dmitriy Pavlov, any ideas? > > D. > > On Wed, Feb 7, 2018 at 9:07 AM, Nikolay Izhikov <nizhi...@apache.org> > wrote: > > > Hello, Igniters. > > > > Please, consider including IGNITE-7337 - Spark Data Frames: support > saving > > a data frame in Ignite [1] in the 2.4 release. > > It seems we can merge it into the master in a day or few. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7337 > > > > > > В Ср, 07/02/2018 в 08:35 -0800, Denis Magda пишет: > > > I’m voting for the blocker addition into the release. Sergey K. how > will > > it affect your testing cycles? Do you need to re-run everything from > > scratch and how many days you need? > > > > > > — > > > Denis > > > > > > > On Feb 6, 2018, at 11:29 PM, Alexey Goncharuk < > > alexey.goncha...@gmail.com> wrote: > > > > > > > > Guys, > > > > > > > > Thanks to Dmitriy Pavlov we found the ticket [1] which causes a major > > > > slowdown when page replacement starts. Even though it's not a > > regression, I > > > > suggest we consider it a blocker for 2.4 because this is a huge > > performance > > > > issue which can make it virtually impossible to use native > persistence > > when > > > > data size is significantly larger than memory size. > > > > > > > > Any objections? > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7638 > > > > > > > > 2018-01-30 17:10 GMT+03:00 Pavel Tupitsyn <ptupit...@apache.org>: > > > > > > > > > Igniters, I will handle 2.4 release if there are no objections. > > > > > Let's take a bit more time for testing and start vote by the end of > > this > > > > > week. > > > > > > > > > > Pavel > > > > > > > > > > On Sat, Jan 27, 2018 at 3:32 AM, Denis Magda <dma...@apache.org> > > wrote: > > > > > > > > > > > Hi Vyacheslav, > > > > > > > > > > > > According to the previous review notes the impact of the changes > > might be > > > > > > significant, thus, I would recommend us to move the changes to > the > > next > > > > > > release. > > > > > > > > > > > > BTW, don’t hesitate to ping reviewers more frequently if there > is a > > > > > > pending/abandon review. We are all the people who are tend to > > forget or > > > > > > miss notifications ;) > > > > > > > > > > > > > On Jan 26, 2018, at 2:04 AM, Vyacheslav Daradur < > > daradu...@gmail.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > Hi, Vladimir, it's good news. I'm looking forward to new Ignite > > > > > > > > > > release! > > > > > > > > > > > > > > Could you please share a release schedule for 'varint' > > optimizations? > > > > > > > > > > > > > > The task [1] is waiting for review for 5 months. > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5097 > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 26, 2018 at 12:51 PM, Vladimir Ozerov < > > > > > > > > > > voze...@gridgain.com> > > > > > > wrote: > > > > > > > > Hi Igniters, > > > > > > > > > > > > > > > > As far as I can see all required tasks and fixes were > merged. I > > > > > > > > > > propose > > > > > > to > > > > > > > > take several days of silence to test what we've done and > start > > vote at > > > > > > > > > > > > the > > > > > > > > beginning of the next week. > > > > > > > > > > > > > > > > Makes sense? > > > > > > > > > > > > > > > > On Mon, Jan 22, 2018 at 8:39 PM, Denis Magda < > > dma...@apache.org> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Ok, let’s target Wednesday as a code freeze date. > > > > > > > > > > > > > > > > > > Community members who are involved in 2.4 release please > > merge you > > > > > > > > > > > > fixes > > > > > > > > > and optimizations by that time. > > > > > > > > > > > > > > > > > > — > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > On Jan 22, 2018, at 8:24 AM, Anton Vinogradov < > > a...@apache.org> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Issues > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6711 > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-6902 > > > > > > > > > > > > > > > > > > > > are in master and ignite-2.4 > > > > > > > > > > > > > > > > > > > > On Mon, Jan 22, 2018 at 6:43 PM, Denis Magda < > > dma...@apache.org> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > > What’s left and what prevents us from passing 2.4 > > through the QA > > > > > > > > > > > > phase? > > > > > > > > > > > > > > > > > > > > > > Petr, Anton, Vladimir, Alex G., others please chime in. > > > > > > > > > > > > > > > > > > > > > > — > > > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > On Jan 17, 2018, at 7:05 PM, Dmitriy Setrakyan < > > > > > > > > > > > > dsetrak...@apache.org> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Great news, indeed! Looking forward to 2.4 release. > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jan 17, 2018 at 2:55 AM, Vladimir Ozerov < > > > > > > > > > > > > voze...@gridgain.com > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > > > > > > Great news - two very important tickets - baseline > > topology and > > > > > > > > > > > > Java 8 > > > > > > > > > > > > > support - were merged to master. At this point it > > makes sense to > > > > > > > > > > > > > > > > > > create > > > > > > > > > > > a > > > > > > > > > > > > > branch to stabilize the release and finalize > > outstanding > > > > > > > > > > tickets. I > > > > > > > > > > > created > > > > > > > > > > > > > the branch *ignite-2.4*. Please follow the rules > > below when > > > > > > > > > > merging > > > > > > > > > new > > > > > > > > > > > > > commits to master: > > > > > > > > > > > > > 1) If ticket is targeted for 2.4 release, please > > merge to master, > > > > > > > > > > > > then > > > > > > > > > > > > > cherry-pick to ignite-2.4 > > > > > > > > > > > > > 2) Otherwise just merge to master. > > > > > > > > > > > > > > > > > > > > > > > > > > Vladimir. > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jan 16, 2018 at 12:32 PM, Anton Vinogradov > < > > > > > > > > > > > > > avinogra...@gridgain.com > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > Denis, > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'll review https://issues.apache.org/ > > jira/browse/IGNITE-6902 > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 11:45 PM, Denis Magda < > > > > > > > > > > dma...@apache.org> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Here is the page create before: > > > > > > > > > > > > > > > https://cwiki.apache.org/ > > confluence/display/IGNITE/ > > > > > > > > > > > > > > > > > > Apache+Ignite+2.4 > > > > > > > > > > > < > > > > > > > > > > > > > > > https://cwiki.apache.org/ > > confluence/display/IGNITE/ > > > > > > > > > > > > > > > > > > Apache+Ignite+2.4> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is good news. It will be perfect if we > also > > release as > > > > > > > > > > many > > > > > > > > > > > > > > "required > > > > > > > > > > > > > > > tickets" as we can: > > > > > > > > > > > > > > > https://cwiki.apache.org/ > > confluence/display/IGNITE/ > > > > > > > > > > > > > > > > > > Apache+Ignite+2.4 > > > > > > > > > > > < > > > > > > > > > > > > > > > https://cwiki.apache.org/ > > confluence/display/IGNITE/ > > > > > > > > > > > > > > > > > > Apache+Ignite+2.4> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > My favorite is memory metrics in bytes: > > > > > > > > > > > > https://issues.apache.org/ > > > > > > > > > > > > > > > jira/browse/IGNITE-6902 < > > https://issues.apache.org/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > jira/browse/IGNITE-6902 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The discussion lists are flooded with the > > demands for this > > > > > > > > > > > > essential > > > > > > > > > > > > > > > capability. Who is going to review it? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > - > > > > > > > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Jan 12, 2018, at 11:22 AM, Dmitriy > > Setrakyan < > > > > > > > > > > > > > > > > > > > > > > > > > > dsetrak...@apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do we have a list of features for 2.4 > > documented anywhere? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > D. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 7:43 AM, Pavel > > Tupitsyn < > > > > > > > > > > > > > > > > > > > > > > > > > > ptupit...@apache.org> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > But some stuff is not yet in master (like > > baseline topology), > > > > > > > > > > > > so > > > > > > > > > it > > > > > > > > > > > > > is > > > > > > > > > > > > > > > too > > > > > > > > > > > > > > > > > soon to create a branch, I think. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Pavel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 12, 2018 at 5:31 PM, Vladimir > > Ozerov < > > > > > > > > > > > > > > > > > > > > > > > > > > > > voze...@gridgain.com> > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > A number of major improvements have been > > made (or being > > > > > > > > > > > > finalized > > > > > > > > > > > > > at > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > moment) to Ignite since recent 2.3 > > release. This includes > > > > > > > > > > > > > > > > > > baseline > > > > > > > > > > > > > > > > > > topology, new DDL commands, migration to > > Java 8 and Java 9 > > > > > > > > > > > > > > > > > > support, > > > > > > > > > > > > > > > > > > critical performance improvements to WAL, > > etc.. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think it makes sense to make a new > > release. What do you > > > > > > > > > > > > think > > > > > > > > > if > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > > release AI 2.4 with all this great stuff > > by the end of Jan? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If there are no objections, I'll create a > > branch to start > > > > > > > > > > > > > > > > > > > > > > > > > > > > stabilization > > > > > > > > > > > > > > > > > > process. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Vladimir. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best Regards, Vyacheslav D. > > > > > > > > > > > > > > > > > > > > >
