Re: Durable Memory and Ignite Persistence Tuning

2017-09-15 Thread Denis Magda
Thanks Ivan, *no less* sounds best for me. Prachi, please do final editing of the doc: https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning — Denis > On Sep 15, 2017, at 12:24 AM, Ivan Rakov wrote: > > Denis, > > Yes, Ignite page size should be *greater or equal* than OS page cache

Re: Durable Memory and Ignite Persistence Tuning

2017-09-15 Thread Ivan Rakov
Denis, Yes, Ignite page size should be *greater or equal* than OS page cache size and SSD page size. I mentioned it in advice list: page size of Ignite shouldn't be less than SSD page size Page size of Ignite shouldn't be less than OS page size Sorry for vague wording. We should fix this in

Re: Durable Memory and Ignite Persistence Tuning

2017-09-14 Thread Denis Magda
Igniters, The first version of the guide is ready. Please check it up before it’s shared with the users: https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning *Ivan*, please clarify this: - Should Ignite page size be always less than the SSD’s page size and OS page cache size? Or it c

Re: Durable Memory and Ignite Persistence Tuning

2017-09-13 Thread Denis Magda
Ivan, Documented: https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning However, I’m a bit confused with you 6. point regarding the page size calculation. Should Ignite page size be always less than the SSD’s page size a

Re: Durable Memory and Ignite Persistence Tuning

2017-09-13 Thread Denis Magda
That’s exactly the discussion around that documentation. Feel free to add these useful points there or wait while I’ll do this later. — Denis > On Sep 13, 2017, at 12:27 PM, Dmitriy Setrakyan wrote: > > Completely agree with Nikita. Why not add this information here? > > https://apacheignite.

Re: Durable Memory and Ignite Persistence Tuning

2017-09-13 Thread Dmitriy Setrakyan
Completely agree with Nikita. Why not add this information here? https://apacheignite.readme.io/docs/durable-memory-tuning D. On Wed, Sep 13, 2017 at 2:54 AM, Nikita Ivanov wrote: > Excellent work on this... This should be expanded and be prominently placed > in our docs/tutorials/javadocs/etc

Re: Durable Memory and Ignite Persistence Tuning

2017-09-13 Thread Nikita Ivanov
Excellent work on this... This should be expanded and be prominently placed in our docs/tutorials/javadocs/etc. -- Nikita Ivanov On Wed, Sep 13, 2017 at 12:29 PM, Ivan Rakov wrote: > Folks, > > We had some experience of benchmarking Ignite with persistent store on > SSD. I think we can share s

Re: Durable Memory and Ignite Persistence Tuning

2017-09-13 Thread Ivan Rakov
> None of them require changing configuration of Ignite or persistent store Disclaimer: this will be true since 2.3. https://issues.apache.org/jira/browse/IGNITE-6182 - memory policy default max size set to 20% of RAM - since 2.2 https://issues.apache.org/jira/browse/IGNITE-5884 - default pageSi

Re: Durable Memory and Ignite Persistence Tuning

2017-09-13 Thread Ivan Rakov
Folks, We had some experience of benchmarking Ignite with persistent store on SSD. I think we can share some helpful advice. None of them require changing configuration of Ignite or persistent store. *Tuning advice for users* 1) Be prepared for LFS performance decrease after several hours of

Re: Durable Memory and Ignite Persistence Tuning

2017-09-01 Thread dsetrakyan
Cos, great point! I think the Ignite community should start load testing with default settings, without any config changes. This should open a lot of holes. ⁣D.​ On Sep 1, 2017, 9:57 PM, at 9:57 PM, Konstantin Boudnik wrote: >Just to spice it up: in my experience, having a few hundred paramete

Re: Durable Memory and Ignite Persistence Tuning

2017-09-01 Thread Nikita Ivanov
My points was in support of auto-tune for Ignite's internals. As far as environment, sure, we need to have a doc. -- Nikita Ivanov On Fri, Sep 1, 2017 at 1:23 PM, Denis Magda wrote: > It doesn’t matter how many configuration parameters your platform, > database or operating system has - the per

Re: Durable Memory and Ignite Persistence Tuning

2017-09-01 Thread Denis Magda
It doesn’t matter how many configuration parameters your platform, database or operating system has - the performance tuning usually takes place in business deployments. The performance optimizations might happen behind the scene by heuristic algorithms or basic checks or be explained in perform

Re: Durable Memory and Ignite Persistence Tuning

2017-09-01 Thread Nikita Ivanov
200% agree. UX is major problem for Ignite (especially so for 2.0 with all major redev). Removing (!) configuration properties should be THE GOAL, not adding new one or documenting them (which is a crutch to begin with). When I see a product with dozens config properties or more I desperately loo

Re: Durable Memory and Ignite Persistence Tuning

2017-09-01 Thread Denis Magda
Cos, Sure, as I see from the @dev list a lot of efforts are put to the auto-tune way. We need to keep to this way whenever it’s possible. However, it might be the case that we can’t auto-tune some parameters (Linux kernel behavior) or we know that setting A will be auto-tuned in release 2.x bu

Re: Durable Memory and Ignite Persistence Tuning

2017-09-01 Thread Konstantin Boudnik
Just to spice it up: in my experience, having a few hundred parameters one can tweak (I am making up the number here) is a tough UX call. In JVM, we had a team that worked on heuristics that would auto-tune a bunch of things during the VM startup. Hence, providing the best user experience in most c

Durable Memory and Ignite Persistence Tuning

2017-09-01 Thread Denis Magda
Igniters, I see a lot of complains regarding the performance of the subj on the user list. At the same time, I do believe that in most scenarios it’s a lack of knowledge that we keep in secret. It's time to document Durable Memory and its Native Persistence tuning parameters. Let's start doing