According to monitoring, there's no increase in use of file handles or file descriptors in the period of heavy load, on the entire system.
On Sun, Jul 23, 2023 at 3:30 PM ufuk yılmaz <uyil...@vivaldi.net.invalid> wrote: > Can it be related to file descriptor/open file handles limit? > > — > > > On 23 Jul 2023, at 14:24, Koen De Groote <kdg....@gmail.com> wrote: > > > > Shawn, > > > > After having a look at these files: No, I cannot share them. > > > > What I can say is that there's a couple hundred fields, dynamicFields and > > copyFields(each). > > > > The updatehandler uses solr.DirectUpdateHandler2(the only one I can see > in > > the source code extending the regular updateHandler), with a max > autoCommit > > time of 60000 and a max autoSoftCommit time of 1000 > > > > Regards, > > Koen > > > > > >> On Sun, Jul 23, 2023 at 1:43 AM Shawn Heisey <elyog...@elyograg.org> > wrote: > >> > >>> On 7/22/23 17:09, Koen De Groote wrote: > >>> Recently, I experienced softCommits taking up to 30 seconds to return. > >> The > >>> application in question was creating a document per interaction and > >> doing a > >>> soft commit at the end of the interaction. After a period of a few > dozens > >>> clients sending a continuous stream of such interactions, I could see > it > >>> getting slower and slower and the source appears to be the softCommit. > >>> > >>> No changes in the XML config have been provided in terms of commit > >> timings. > >>> The JVM is given 20GB heap space, of which it seems to hang steady at > >> 10GB > >>> at all times throughout usage, and there's some 25M documents getting > up > >> to > >>> a total of 150GB of data on disk. Everything is on 1 shard, with 2 > hosts > >>> each having 1 instance of the collection. The underlying disk is an SSD > >> on > >>> both hosts. > >>> > >>> Before I dive into documentation or code, I was wondering if anyone > here > >>> might have immediate ideas of what could cause such behavior for soft > >>> commits. > >>> > >>> If someone has an immediate bit of knowledge towards what causes > >>> softCommits to take up to 30 seconds, that'd be appreciated. > >> > >> Can you share the whole config -- solrconfig.xml, the schema, and any > >> file(s) referenced by either of those. The schema may be named > >> managed-schema.xml, managed-schema, or schema.xml (or even something > >> different) depending on Solr version and the rest of the config. > >> > >> Normally there isn't anything sensitive in these files, but if you do > >> have something, redact it as minimally as possible, don't just delete > >> the whole section with the sensitive data. > >> > >> Thanks, > >> Shawn > >> > >
