Apache flume did those kinds of things via Solr sink, not sure if it’s still supported since it’s a bit old.
— > On May 13, 2025, at 21:22, Thomas Corthals <tho...@klascement.net> wrote: > > Hi Dario, > > Regardless of this GOAWAY signal, I would advise to buffer the documents on > the client side instead of indexing them individually for every event. > You'll have to try out which numbers work for your specific document size > and timeliness expectations. I usually start with a buffer size of 200 > documents, could be higher for very small documents or lower for very large > ones. When that amount is reached, flush them to Solr in a single request. > If you can't have documents show up "late", you can additionally flush your > buffer after e.g. 1 minute even if it has fewer documents. > > Thomas > > Op di 13 mei 2025 om 11:02 schreef <dario.v...@coop.ch.invalid>: > >> Dear Solr People >> >> On our system we have an indexer-service that indexes documents based on >> incoming events. Sometimes a lot of these events happen at the same time >> (or at least very close in time to each other). If that happens our indexer >> receives GOAWAY signals from solr. >> We're using the HttpJdkSolrClient with HTTP/2. Solr Version is 9.5.0 >> >> When exactly do these GOAWAY signals show up. In other words, how many >> requests need to be made to solr in a certain amount of timespan that solr >> starts emitting this signal. And how exactly should it be handled or fixed. >> >> Should we switch to HTTP/1.1? This version of HTTP does not specify this >> signal, but this is probably not a solution? >> Using another client? >> >> Can there be done anything else about it? Will the documents that were in >> the request that caused the GOAWAY signal still get added to the index or >> do we need to resend this request? And If yes, how long should we wait to >> resend it? >> >> With kind regards, >> >> Dario >>
