Folks, I keep working on crash recovery speed-up.
The main goal is to have put/get operations latency less than 500 ms on node fail/left. Currently, latency can be increased to seconds or even dozens of seconds. The task is split to 2 threads - Switch and tx recovery speed-up. Speed-up can be gained by code refactoring, simplification, and tuning as well as by special tricks like pme-free switch or cellular affinity usage. - Failure detection. Already found that some constants used at failure detection are hardcoded and large. Also, code responsible for this feature performs a lot of re-checks and re-waits and you may have detection time close to failureDetectionTimeout x2 or even x3. Another problem is GC, and it may increase failure detection dramatically, so, watchdog started from another JVM or from native code can help here. On Tue, Apr 28, 2020 at 2:13 AM Denis Magda <dma...@apache.org> wrote: > Nikolay, thanks for responding, > > Would you prefer adding this item to the roadmap page after finishing the > idea testing? If there are still many unknowns it can make to the roadmap > after you share the results and propose a final solution. > > - > Denis > > > On Mon, Apr 27, 2020 at 2:06 PM Nikolay Izhikov <nizhi...@apache.org> > wrote: > > > Hello, Igniters. > > > > Right now I working on POC for some «framework» for integration testing > of > > Ignite. > > > > Feature I want to see in this framework: > > > > * To manage several Ignite instances inside docker or bare metal > > servers. > > * To manage Ignite cluster > > * Start/stop arbitrary application to interact with Ignite. > > * Kill nodes > > * collect logs. > > * implement complex real-life based scenarios > > * etc. > > > > My POC are based on duck tape library [1] > > > > [1] https://github.com/confluentinc/ducktape > > > > > 25 апр. 2020 г., в 00:51, Denis Magda <dma...@apache.org> написал(а): > > > > > > Dmitry, highlighted such a disclaimer in italic. Thanks. > > > > > > - > > > Denis > > > > > > > > > On Thu, Apr 23, 2020 at 3:53 AM Dmitriy Pavlov <dpav...@apache.org> > > wrote: > > > > > >> Hi Denis, > > >> > > >> Thank you for driving this. > > >> > > >> Igniters, > > >> > > >> I would suggest to stress that Apache Ignite community does not > > guarantee > > >> these features to be available. > > >> > > >> Can we add some kind of disclaimer that says Ignite Roadmap does not > > imply > > >> any obligations regarding availability and timeline. A number of > > >> contributions can be done on best efforts principle, it is always > > tricky to > > >> make a promise. > > >> > > >> Sincerely, > > >> Dmitriy Pavlov > > >> > > >> чт, 23 апр. 2020 г. в 00:06, Denis Magda <dma...@apache.org>: > > >> > > >>> Igniters, > > >>> > > >>> Here is a draft of our very first roadmap. I decided to make it damp > > >> simple > > >>> but descriptive: > > >>> > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap > > >>> > > >>> What we need to do next is to: > > >>> > > >>> - Fill in the "Readiness Estimated Time" column with your best > guess > > >> of > > >>> when an improvement is to be ready for a release. > > >>> - Add references to JIRAs or IEPs to the first column. > > >>> - Add the names of those contributors who will be cooperating with > > you > > >>> during development. Goes to the "Contributors" column. > > >>> > > >>> Once this step is complete, we'll see how many features converge > around > > >> the > > >>> same date/months and we'll plan through 2.9, 2.10, etc. releases > > >>> accordingly. > > >>> > > >>> Please don't put this aside for a while, let's move on quicker. If > the > > >>> roadmap misses any contributions that you are going to add, then edit > > the > > >>> wiki page. > > >>> > > >>> - > > >>> Denis > > >>> > > >>> > > >>> On Wed, Apr 15, 2020 at 3:35 AM Nikita Amelchev < > nsamelc...@gmail.com> > > >>> wrote: > > >>> > > >>>> Hello, Igniters. > > >>>> > > >>>> I am going to contribute a new feature - profiling tool and > > >>>> performance report. This is part of IEP-35. [1] > > >>>> > > >>>> The tool will be able to collect performance statistics and create a > > >>>> human-readable report. It will help to analyze workload and to tune > > >>>> configuration and applications. > > >>>> > > >>>> Example of report [2, 3, 4]. > > >>>> > > >>>> [1] > > >>>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool > > >>>> [2] > > >>>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png > > >>>> [3] > > >>>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png > > >>>> [4] > > >>>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png > > >>>> > > >>>> сб, 11 апр. 2020 г. в 13:54, Alexei Scherbakov < > > >>>> alexey.scherbak...@gmail.com>: > > >>>>> > > >>>>> Folks, > > >>>>> > > >>>>> I keep working on tasks related to data consistency. > > >>>>> > > >>>>> This includes: > > >>>>> > > >>>>> Lost partitions handling overhaul (almost done) and tombstones > > >> support > > >>>>> aforementioned by Ivan Rakov > > >>>>> Atomic protocol overhaul (see [1]) > > >>>>> > > >>>>> The ultimate goal of the year is to prepare Ignite for passing > Jepsen > > >>>> tests. > > >>>>> > > >>>>> [1] > > >>>>> > > >>>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-12+Make+ATOMIC+Caches+Consistent+Again > > >>>>> > > >>>>> > > >>>>> пт, 10 апр. 2020 г. в 18:49, Denis Magda <dma...@apache.org>: > > >>>>> > > >>>>>> Steven, > > >>>>>> > > >>>>>> Please start a dedicated discussion for the Golang support. At the > > >>>> moment, > > >>>>>> I'm not aware if anybody from the community planned to provide > > >>> support > > >>>>>> out-of-the-box. However, that's not a tricky task thanks to > > >> Ignite's > > >>>> binary > > >>>>>> protocol that lets enable any programming language support easily. > > >>>>>> > > >>>>>> - > > >>>>>> Denis > > >>>>>> > > >>>>>> > > >>>>>> On Fri, Apr 10, 2020 at 8:43 AM smeadows-abb < > > >>>> steven.mead...@us.abb.com> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> First thanks for your quick response. > > >>>>>>> > > >>>>>>> I looked at https://github.com/amsokol/ignite-go-client and its > > >>> NOT > > >>>>>>> completed and nothing has been for last 16 months. Initial test > > >>> with > > >>>>>>> package > > >>>>>>> failed, so trying to determine your project roadmap with regards > > >> to > > >>>>>> Golang > > >>>>>>> and maybe Rust support. > > >>>>>>> > > >>>>>>> I'm NOT sure of 'AFAIK' ? > > >>>>>>> > > >>>>>>> We may need to implement your Restful API to provide support for > > >>>> Golang > > >>>>>> and > > >>>>>>> Rust, provided it's complete? > > >>>>>>> > > >>>>>>> Thanks, > > >>>>>>> Steve > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> -- > > >>>>>>> Sent from: > > >> http://apache-ignite-developers.2346864.n4.nabble.com/ > > >>>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> > > >>>>> Best regards, > > >>>>> Alexei Scherbakov > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> Best wishes, > > >>>> Amelchev Nikita > > >>>> > > >>> > > >> > > > > >
