On 3/8/2021 8:18 PM, Hitesh Khamesra wrote:
We are getting OutOfMemoryError, while concurrently deleting and updating
the data. Looks like we don't have a thread pool to merge segment/index. Is
it possible to control this behavior?
Here's the important part from that massive exception stacktrace:
Caused by: java.lang.OutOfMemoryError: unable to create native thread:
There are exactly two ways you can fix OOME. One is to increase the
resource that has been depleted, the other is to change things so that
fewer resources are required.
In this case you're running into an OS limit on the number of processes
that a user is allowed to start. The default setting for this limit on
all Linux distros I have seen is 1024. It's pretty easy to have a Solr
instance with significantly more than a thousand threads. On most
operating systems, threads are handled by the kernel as processes. Some
have separate entities for threads, but that's not very common.
So what you're going to have to do is increase the number of processes
that the user which is running Solr is allowed to have.
On most Linux distros this is usually done with the following file:
/etc/security/limits.conf
If you're running with a different OS, you'll need to research how to
increase the limit.
Thanks,
Shawn
