Thanks, Shwan. We checked our limits and those looked high enough. I am wondering if there is any possibility to introduce a thread pool for segment merge above?
Also, do we know what is the appropriate size for jvm stack size (Xss) Thanks. Hitesh. On Mon, Mar 8, 2021 at 10:33 PM Shawn Heisey <apa...@elyograg.org> wrote: > On 3/8/2021 8:18 PM, Hitesh Khamesra wrote: > > We are getting OutOfMemoryError, while concurrently deleting and updating > > the data. Looks like we don't have a thread pool to merge segment/index. > Is > > it possible to control this behavior? > > Here's the important part from that massive exception stacktrace: > > Caused by: java.lang.OutOfMemoryError: unable to create native thread: > > There are exactly two ways you can fix OOME. One is to increase the > resource that has been depleted, the other is to change things so that > fewer resources are required. > > In this case you're running into an OS limit on the number of processes > that a user is allowed to start. The default setting for this limit on > all Linux distros I have seen is 1024. It's pretty easy to have a Solr > instance with significantly more than a thousand threads. On most > operating systems, threads are handled by the kernel as processes. Some > have separate entities for threads, but that's not very common. > > So what you're going to have to do is increase the number of processes > that the user which is running Solr is allowed to have. > > On most Linux distros this is usually done with the following file: > > /etc/security/limits.conf > > If you're running with a different OS, you'll need to research how to > increase the limit. > > Thanks, > Shawn >
