Can it be related to file descriptor/open file handles limit? —
> On 23 Jul 2023, at 14:24, Koen De Groote <kdg....@gmail.com> wrote: > > Shawn, > > After having a look at these files: No, I cannot share them. > > What I can say is that there's a couple hundred fields, dynamicFields and > copyFields(each). > > The updatehandler uses solr.DirectUpdateHandler2(the only one I can see in > the source code extending the regular updateHandler), with a max autoCommit > time of 60000 and a max autoSoftCommit time of 1000 > > Regards, > Koen > > >> On Sun, Jul 23, 2023 at 1:43 AM Shawn Heisey <elyog...@elyograg.org> wrote: >> >>> On 7/22/23 17:09, Koen De Groote wrote: >>> Recently, I experienced softCommits taking up to 30 seconds to return. >> The >>> application in question was creating a document per interaction and >> doing a >>> soft commit at the end of the interaction. After a period of a few dozens >>> clients sending a continuous stream of such interactions, I could see it >>> getting slower and slower and the source appears to be the softCommit. >>> >>> No changes in the XML config have been provided in terms of commit >> timings. >>> The JVM is given 20GB heap space, of which it seems to hang steady at >> 10GB >>> at all times throughout usage, and there's some 25M documents getting up >> to >>> a total of 150GB of data on disk. Everything is on 1 shard, with 2 hosts >>> each having 1 instance of the collection. The underlying disk is an SSD >> on >>> both hosts. >>> >>> Before I dive into documentation or code, I was wondering if anyone here >>> might have immediate ideas of what could cause such behavior for soft >>> commits. >>> >>> If someone has an immediate bit of knowledge towards what causes >>> softCommits to take up to 30 seconds, that'd be appreciated. >> >> Can you share the whole config -- solrconfig.xml, the schema, and any >> file(s) referenced by either of those. The schema may be named >> managed-schema.xml, managed-schema, or schema.xml (or even something >> different) depending on Solr version and the rest of the config. >> >> Normally there isn't anything sensitive in these files, but if you do >> have something, redact it as minimally as possible, don't just delete >> the whole section with the sensitive data. >> >> Thanks, >> Shawn >>
