Point taken. Going over the code, I am seeing *autowarm*Count="0" a few times in the config XML near various LRU and fastLRU cache definitions. Not seeing specific queries defined in any XML.
Regards, Koen On Sun, Jul 23, 2023 at 7:10 PM Rahul Goswami <rahul196...@gmail.com> wrote: > “The application in question was creating a document per interaction and > doing a > soft commit at the end of the interaction.“ > > You also mentioned your autoSoftCommit interval is 1 sec. If you really > need NRT, I would suggest the client stop sending a softCommit upon each > insert since the (extremely) short autoSoftCommit interval is anyway taking > care of making the writes available immediately . > > If that is not possible, try increasing the autoSoftCommit interval in > solrconfig. You don’t need both. Also, do you have any autoWarm cache > queries? > > -Rahul > > > > On Sun, Jul 23, 2023 at 1:00 PM Koen De Groote <kdg....@gmail.com> wrote: > > > According to monitoring, there's no increase in use of file handles or > file > > descriptors in the period of heavy load, on the entire system. > > > > On Sun, Jul 23, 2023 at 3:30 PM ufuk yılmaz <uyil...@vivaldi.net.invalid > > > > wrote: > > > > > Can it be related to file descriptor/open file handles limit? > > > > > > — > > > > > > > On 23 Jul 2023, at 14:24, Koen De Groote <kdg....@gmail.com> wrote: > > > > > > > > Shawn, > > > > > > > > After having a look at these files: No, I cannot share them. > > > > > > > > What I can say is that there's a couple hundred fields, dynamicFields > > and > > > > copyFields(each). > > > > > > > > The updatehandler uses solr.DirectUpdateHandler2(the only one I can > see > > > in > > > > the source code extending the regular updateHandler), with a max > > > autoCommit > > > > time of 60000 and a max autoSoftCommit time of 1000 > > > > > > > > Regards, > > > > Koen > > > > > > > > > > > >> On Sun, Jul 23, 2023 at 1:43 AM Shawn Heisey <elyog...@elyograg.org > > > > > wrote: > > > >> > > > >>> On 7/22/23 17:09, Koen De Groote wrote: > > > >>> Recently, I experienced softCommits taking up to 30 seconds to > > return. > > > >> The > > > >>> application in question was creating a document per interaction and > > > >> doing a > > > >>> soft commit at the end of the interaction. After a period of a few > > > dozens > > > >>> clients sending a continuous stream of such interactions, I could > see > > > it > > > >>> getting slower and slower and the source appears to be the > > softCommit. > > > >>> > > > >>> No changes in the XML config have been provided in terms of commit > > > >> timings. > > > >>> The JVM is given 20GB heap space, of which it seems to hang steady > at > > > >> 10GB > > > >>> at all times throughout usage, and there's some 25M documents > getting > > > up > > > >> to > > > >>> a total of 150GB of data on disk. Everything is on 1 shard, with 2 > > > hosts > > > >>> each having 1 instance of the collection. The underlying disk is an > > SSD > > > >> on > > > >>> both hosts. > > > >>> > > > >>> Before I dive into documentation or code, I was wondering if anyone > > > here > > > >>> might have immediate ideas of what could cause such behavior for > soft > > > >>> commits. > > > >>> > > > >>> If someone has an immediate bit of knowledge towards what causes > > > >>> softCommits to take up to 30 seconds, that'd be appreciated. > > > >> > > > >> Can you share the whole config -- solrconfig.xml, the schema, and > any > > > >> file(s) referenced by either of those. The schema may be named > > > >> managed-schema.xml, managed-schema, or schema.xml (or even something > > > >> different) depending on Solr version and the rest of the config. > > > >> > > > >> Normally there isn't anything sensitive in these files, but if you > do > > > >> have something, redact it as minimally as possible, don't just > delete > > > >> the whole section with the sensitive data. > > > >> > > > >> Thanks, > > > >> Shawn > > > >> > > > > > > > > >
