The suggestion here is based on initial settings, and it's so because there is no other nodes on the host in the example.
The algorithm tries to preserve the original ratio of memory policies keeping numbers reasonable (for example after some thresshold it will suggest not to use several memory policies if there is not enough of memory for all of them) and taking into consideration nodes count on the host, each jvm heap, needed memory for OS, etc 2017-08-28 14:38 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>: > Looks good, but why in the example provided are we suggesting 8GB? 2 nodes > with 8GB will completely exhaust the available memory. I would suggest 6 or > 7GB. > > Also, why 100MB for default policy. Anything under 1GB seems too small. > > Can you please comment? > > D. > > On Mon, Aug 28, 2017 at 3:31 AM, Seliverstov Igor <gvvinbl...@gmail.com> > wrote: > > > One more example of possible warning: > > > > ----------------------------------------------------- > > Excessive memory usage by Ignite node process (performance may drop) > > [requested=44613MB, available=15942MB]. > > > > Please tune the folowing settings as suggested: > > MemoryPolicyConfiguration.initialSize for bigPlc: 8102MB > > MemoryPolicyConfiguration.maxSize for bigPlc: 8102MB > > MemoryPolicyConfiguration.initialSize for dfltPlc: 100MB > > MemoryPolicyConfiguration.maxSize for dfltPlc: 100MB > > > > Current settings: > > Java Heap maxSize: 3543MB > > Java Heap initSize: 250MB > > MemoryPolicyConfiguration.initialSize for bigPlc: 256MB > > MemoryPolicyConfiguration.maxSize for bigPlc: 40960MB > > MemoryPolicyConfiguration.initialSize for dfltPlc: 10MB > > MemoryPolicyConfiguration.maxSize for dfltPlc: 10MB > > The overall expected memory usage by all Ignite nodes on the host: > > 44613MB > > ----------------------------------------------------- > > > > Your thoughts? > > > > 2017-08-28 5:06 GMT+03:00 Denis Magda <dma...@apache.org>: > > > > > Guys, > > > > > > ML lib profile is missing in 2.1 release! That must be fixed and rolled > > > out in this emergency release: > > > https://issues.apache.org/jira/browse/IGNITE-6193 < > > > https://issues.apache.org/jira/browse/IGNITE-6193> > > > > > > Oleg, Yuri, please step in and handle the issue. > > > > > > BTW, who is considered to be the release manager of this release? > > > > > > — > > > Denis > > > > > > > On Aug 25, 2017, at 2:29 PM, Dmitriy Setrakyan < > dsetrak...@apache.org> > > > wrote: > > > > > > > > I like the format proposed by Denis, very clear. > > > > > > > > However, I also do not understand why a user should change the size > of > > > some > > > > system cache. How would a user ever know what value to put there? > This > > > > value should be configured by Ignite automatically. > > > > > > > > D. > > > > > > > > On Fri, Aug 25, 2017 at 2:24 PM, Denis Magda <dma...@apache.org> > > wrote: > > > > > > > >> Igor, > > > >> > > > >> Let me suggest this format. > > > >> > > > >> --------------------------------------------- > > > >> Excessive memory usage by Ignite node process (performance may drop) > > > >> [requested=29251MB, available=15942MB] > > > >> > > > >> Please tune the following settings: > > > >> [MemoryConfiguration.defaultMemoryPolicySize = suggested value] > > > >> MemoryConfiguration.systemCacheMaxSize = suggested value > > > >> [MemoryPolicyConfiguration.maxSize for {policy_name_1} = suggested > > > >> value] > > > >> [MemoryPolicyConfiguration.maxSize for {policy_name_2} = suggested > > > >> value] > > > >> > > > >> Current settings: > > > >> [DefaultMemoryPolicySize = value] > > > >> [{policy_name_1} size = value] > > > >> [{policy_name_1} size = value] > > > >> SystemCacheInitialSize = value > > > >> SystemCacheMaxSize = value > > > >> Java Heap Init Size = value > > > >> Java Heap Max Size = value > > > >> > > > >> The overall memory usage by all Ignite nodes on the host: value > > > >> ------------------------------------------- > > > >> > > > >> Records in […] are optional. If custom memory policy is not set or > the > > > >> default memory policy is overridden the output will miss some of the > > > rows. > > > >> > > > >> As for systemCacheMaxSize, it should be show ONLY if the parameter > was > > > set > > > >> explicitly by user code. Otherwise, the platform should be wise > enough > > > to > > > >> instantiate it properly depending on the host memory usage. > > > >> > > > >> — > > > >> Denis > > > >> > > > >>> On Aug 25, 2017, at 1:49 PM, Seliverstov Igor < > gvvinbl...@gmail.com> > > > >> wrote: > > > >>> > > > >>> The message without logging layout: > > > >>> > > > >>> Not enough memory for current process [required=29251MB, > > > >> available=15942MB]. > > > >>> Please change MemoryConfiguration.systemCacheMaxSize and > > > >>> MemoryConfiguration.defaultMemoryPolicySize to decrease memory > > > allocated > > > >>> for each node. > > > >>> > > > >>> Current settings: > > > >>> HeapInit=250MB > > > >>> HeapMax=3543MB > > > >>> DefaultMemoryPolicySize=12753MB > > > >>> SystemCacheInitialSize=40MB > > > >>> SystemCacheMaxSize=100MB > > > >>> > > > >>> Other ignite instanses on the server require: 12853MB > > > >>> > > > >>> I think it's make sense to describe what these numbers are consist > > of. > > > >>> > > > >>> We simple say which parameters have an impact on how much memory > the > > > >>> instance needs and their (parameters) actual values. > > > >>> > > > >>> Also we notice that more than one Ignite instance are ran on the > > server > > > >> or > > > >>> workstation and it also consumes memory. > > > >>> > > > >>> 25 авг. 2017 г. 21:30 пользователь "Dmitriy Setrakyan" < > > > >>> dsetrak...@apache.org> написал: > > > >>> > > > >>>> Igor, what is this flood of WARN messaging coming after the text? > > Are > > > we > > > >>>> really going to print this whole thing out? > > > >>>> > > > >>>> On Fri, Aug 25, 2017 at 9:49 AM, Seliverstov Igor < > > > gvvinbl...@gmail.com > > > >>> > > > >>>> wrote: > > > >>>> > > > >>>>> This message appears on topology change in case the available > > memory > > > is > > > >>>>> exceeded > > > >>>>> > > > >>>>> 2017-08-25 19:47 GMT+03:00 Seliverstov Igor < > gvvinbl...@gmail.com > > >: > > > >>>>> > > > >>>>>> An example of current impl: > > > >>>>>> > > > >>>>>> > > > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] > > > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] Not enough memory > > for > > > >>>>>> current process [required=29251MB, available=15942MB]. > > > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] Please change > > > >>>>>> MemoryConfiguration.systemCacheMaxSize and MemoryConfiguration. > > > >>>>> defaultMemoryPolicySize > > > >>>>>> to decrease memory allocated for each node. > > > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] > > > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] Current settings: > > > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] HeapInit=250MB > > > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] HeapMax=3543MB > > > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] > > > >>>> DefaultMemoryPolicySize= > > > >>>>>> 12753MB > > > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] > > > >>>>>> SystemCacheInitialSize=40MB > > > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] > > > >>>> SystemCacheMaxSize=100MB > > > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] > > > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] Other ignite > > > instanses > > > >>>> on > > > >>>>>> the server require: 12853MB > > > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29% > internal. > > > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] > > > >>>>>> > > > >>>>>> > > > >>>>>> 2017-08-25 17:40 GMT+03:00 Sergey Kozlov <skoz...@gridgain.com > >: > > > >>>>>> > > > >>>>>>> I suppose we should not forget JVM heap size and suggest to > > reduce > > > >>>> both > > > >>>>>>> options > > > >>>>>>> > > > >>>>>>> On Fri, Aug 25, 2017 at 5:24 PM, Dmitriy Setrakyan < > > > >>>>> dsetrak...@apache.org > > > >>>>>>>> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> Igor, I would change the message. How about this: > > > >>>>>>>> > > > >>>>>>>> Required RAM size is larger than total physical memory > available > > > for > > > >>>>> OS. > > > >>>>>>>>> Please change MemoryConfiguration.WhichProperty and > > > >>>>>>>>> MemoryPolicyConfiguration.WhichProperty to decrease memory > > > >>>>> allocated > > > >>>>>>> for > > > >>>>>>>>> each node. > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> Also, can we calculate what the memory size allocated for each > > > node > > > >>>>>>> should > > > >>>>>>>> be? In that case we should suggest it. > > > >>>>>>>> > > > >>>>>>>> D. > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> On Fri, Aug 25, 2017 at 7:20 AM, Seliverstov Igor < > > > >>>>> gvvinbl...@gmail.com > > > >>>>>>>> > > > >>>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> What do you, guys think about next warning? > > > >>>>>>>>> > > > >>>>>>>>> [2017-08-25 17:17:04,718][INFO > > > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat > > > >>>>>>> hSelfTest0] > > > >>>>>>>>> System cache's MemoryPolicy size is configured to 40 MB. Use > > > >>>>>>>>> MemoryConfiguration.systemCacheMemorySize property to change > > the > > > >>>>>>>> setting. > > > >>>>>>>>> [2017-08-25 17:17:04,718][WARN > > > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat > > > >>>>>>> hSelfTest0] > > > >>>>>>>>> > > > >>>>>>>>>>>> Required RAM size is larger than total physical memory > > > >>>> available > > > >>>>>>> for > > > >>>>>>>>> OS. > > > >>>>>>>>>>>> Check your configuration to avoid swap partition usage. > > > >>>>>>>>>>>> Use MemoryConfiguration and MemoryPolicyConfiguration to > > > >>>> change > > > >>>>>>> the > > > >>>>>>>>> settings. > > > >>>>>>>>>>>> Physical memory [required=16397MB, available=15942MB] > > > >>>>>>>>> > > > >>>>>>>>> [2017-08-25 17:17:04,726][WARN > > > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat > > > >>>>>>> hSelfTest0] > > > >>>>>>>>> Peer class loading is enabled (disable it in production for > > > >>>>>>> performance > > > >>>>>>>> and > > > >>>>>>>>> deployment consistency reasons) > > > >>>>>>>>> [2017-08-25 17:17:04,726][INFO > > > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat > > > >>>>>>> hSelfTest0] > > > >>>>>>>>> Configured caches [in 'sysMemPlc' memoryPolicy: > > > >>>>> ['ignite-sys-cache']] > > > >>>>>>>>> [2017-08-25 17:17:04,731][INFO > > > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat > > > >>>>>>> hSelfTest0] > > > >>>>>>>>> 3-rd party licenses can be found at: > > > >>>>>>>>> /home/gvvinblade/projects/ignite/incubator-ignite/libs/ > > licenses > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> 2017-08-25 13:26 GMT+03:00 Yakov Zhdanov < > yzhda...@apache.org > > >: > > > >>>>>>>>> > > > >>>>>>>>>> Agree, let's release new version including tickets mentioned > > by > > > >>>>>>> Denis > > > >>>>>>>> and > > > >>>>>>>>>> Ivan. > > > >>>>>>>>>> > > > >>>>>>>>>> --Yakov > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> -- > > > >>>>>>> Sergey Kozlov > > > >>>>>>> GridGain Systems > > > >>>>>>> www.gridgain.com > > > >>>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>>> > > > >> > > > >> > > > > > > > > >
