hen you don’t need to pre-touch everything to
> make sure the memory is there
> >
> > - eager region init + pre-touch – to fail as early as possible if there
> is not enough RAM
> >
> >
> >
> > Stan
> >
> >
> >
> > From: Nikolay Izhikov
&g
-touch everything to make
> sure the memory is there
>
> - eager region init + pre-touch – to fail as early as possible if there is
> not enough RAM
>
>
>
> Stan
>
>
>
> From: Nikolay Izhikov
> Sent: 14 декабря 2018 г. 14:42
> To: dev; vololo...@gmail.com;
fail as early as possible if there is not
enough RAM
Stan
From: Nikolay Izhikov
Sent: 14 декабря 2018 г. 14:42
To: dev; vololo...@gmail.com; stanlukya...@gmail.com
Subject: Re: Pre-touch for Ignite off-heap memory
Bump.
Stanislav, Ivan, can you answer my questions?
Let's discuss
its for Ignite because we use
> off-heap as well as heap.
> > > So, we can implement a similar feature for Ignite – and make sure the
> users can get all the memory pre-touch benefits if they want.
> > > Of course, it impacts startup time so we should enable it by a
> configurat
d suggest a system property for now).
> >
> > Are there any objections to implementing this?
> >
> > Thanks,
> > Stan
> >
> > From: Павлухин Иван
> > Sent: 31 октября 2018 г. 12:50
> > To: dev@ignite.apache.org
> > Subject: Re: Pre-touch
Sent: 31 октября 2018 г. 12:50
> To: dev@ignite.apache.org
> Subject: Re: Pre-touch for Ignite off-heap memory
>
> Hi,
>
> I did an experiment described above. I created a patch with pre-touch [1]
> and started a JVM with an option -XX:+AlwaysPreTouch and configured
> Ign
this?
Thanks,
Stan
From: Павлухин Иван
Sent: 31 октября 2018 г. 12:50
To: dev@ignite.apache.org
Subject: Re: Pre-touch for Ignite off-heap memory
Hi,
I did an experiment described above. I created a patch with pre-touch [1]
and started a JVM with an option -XX:+AlwaysPreTouch and configured
Ignite
Hi,
I did an experiment described above. I created a patch with pre-touch [1]
and started a JVM with an option -XX:+AlwaysPreTouch and configured
Ignite with equal values for initial and max sizes for each data region.
I did several runs. I observed JVM crash dumps [2], [3]. Also it is easy
to obs
Hi guys,
I am not aware that it is possible to run JVM in "allocation-free" fashion.
If you know that it is possible please share it. As I know JVM allocates
memory out of garbage collectable area for internal purposes like JIT,
GC itself. Also native built-it code can request memory allocation fr
Andrey,
Probability of a OOM kill will be much lower if offheap is pretouched. What
do you mean by JVM internal needs? In my understanding if user enables
option to pretouch heap and fixes the heap to prevent jvm releasing memory
back to OS, then OOM killing is very unlikely.
I would agree that p
Hi Alex,
1. Pre-touch will not save from killing node by OS if there will not enough
memory for JVM internal needs, anyway.
Ignite data region setting has MIN and MAX sizes. Ignite split memory
region into segments, first segment is size of MIN.
Next segments are allocated on demand (up to MAX) an
In my understanding, that's an easy operation to support - we just need to
allocate requested space and pre-touch it (iterate and fill out with 0).
Yes, it will increase the launch time but it's a valid use case. Some
companies do memory pre-touching on purpose.
--
Denis
On Thu, Oct 25, 2018 at
Hi, Ivan
Let me try to explain. I think that there are 2 goals that are mostly
applicable for prod systems:
1. I can agree with Dave that swap is not a good case for performance and
disk resource. It is possible that other applications can consume memory
that is free after Ignite was started. In th
Hi Alex,
I wonder what problem you wish to solve. Pre-touching memory looks like a
solution.
But what is the problem? Am I getting it right that it is desired to avoid
OOM (of some kind)
in unpredictable moment of Ignite process execution? If so there could be
number of ways to tackle it.
I am no
Denis,
We run must of our production DBs systems without any swapping space,
because the 10-100x drop in throughput if such systems start paging makes
them worse than useless. However, we don't get OOM on them until all the
pages are dirty, since LINUX will page out read-only (code) pages or memo
Alex,
I think offheap memory preallocation will not be helpful as you expect:
1. We still will not have any guarantee JVM process won't be killed by
OOM-killer due to memory overcommiting e.g. for GC needs.
as JVM allocates memory for internal needs in same process memory space.
2. Startup process
Docs [1] says, that OOM can also be thrown when native library can't
allocate memory chunk if physical memory (ram + swap).
>> Shouldn't OOM happen if you're trying to allocate beyond the virtual
memory capacity (beyond X GB)?
With 64-bit addressing you have some exabytes of virtual memory, is it
Alex,
Correct me if I'm wrong, but even if an OS runs out of physical memory (X
GB in total) an Ignite node process still can request the X GB from virtual
memory. Yes, virtual memory can involve swapping and disk to satisfy your
request but this shouldn't be a reason of the OOM. Shouldn't OOM hap
18 matches
Mail list logo
