Have you configured an URL for health check? Which one? The load balancer checks if the solr port is in use or not. If yes, then it continues sending the search requests.
Or you have something like rolling restart/recycle scenarios executed? No, we don't have anything like that configured. @ufuk We may have various scenarios where we have to restart a solr node, like changing heap, gc settings, etc. How can I avoid getting 5xx in these scenarios? On Wed, Jun 14, 2023 at 1:02 PM Mikhail Khludnev <m...@apache.org> wrote: > Saksham, can you comment on > > if a certain port is up or not and based on that send the request to that > node. > > Have you configured an URL for health check? Which one? > > > the coordinator node goes down after a request is sent from lb. > Do you mean nodes are failing more often than healthcheck occur? > > Or you have something like rolling restart/recycle scenarios executed? > > On Wed, Jun 14, 2023 at 8:52 AM Saksham Gupta > <saksham.gu...@indiamart.com.invalid> wrote: > > > @Ufuk We are using a load balancer to avoid a single point of failure > i.e. > > if all the requests have a single coordinator node then it would be a > major > > issue if this solr node goes down. > > > > @Mikhail Khludnev We already have a health check configured on load > > balancer, but the requests will fail if the coordinator node goes down > > after request is sent from lb. > > Further explaining, the load balancer will check if a certain port is up > or > > not and based on that send the request to that node. The issue is > observed > > for cases where the coordinator node goes down after a request is sent > from > > lb. > > > > Please let me know if I am missing something here. Any other > suggestions? > > > > On Tue, Jun 13, 2023 at 7:12 PM Mikhail Khludnev <m...@apache.org> > wrote: > > > > > Well, probably it's what Solr Operator can provide on Kubernetes. > > > > > > On Tue, Jun 13, 2023 at 10:47 AM ufuk yılmaz > <uyil...@vivaldi.net.invalid > > > > > > wrote: > > > > > > > Just wondered, solr cloud itself can handle node failings and load > > > > balancing. Why use an external cloud load balancer? > > > > > > > > —ufuk yilmaz > > > > > > > > — > > > > > > > > > On 13 Jun 2023, at 10:28, Mikhail Khludnev <m...@apache.org> > wrote: > > > > > > > > > > Hello > > > > > You can configure healthcheck > > > > > > > > > > > > > > > https://cloud.google.com/load-balancing/docs/health-check-concepts#criteria-protocol-http > > > > > with Solr's ping request handler > > > > > > https://solr.apache.org/guide/solr/latest/deployment-guide/ping.html > > . > > > > > Also, Google cloud has sophisticated Traffic Director, which can > also > > > > suit > > > > > for node failover. > > > > > > > > > > On Tue, Jun 13, 2023 at 9:13 AM Saksham Gupta > > > > > <saksham.gu...@indiamart.com.invalid> wrote: > > > > > > > > > >> Hi team, > > > > >> We need help with the strategy used to request data from solr > cloud. > > > > >> > > > > >> *Current Searching Strategy:* > > > > >> We are using solr cloud 8.10 having 8 nodes with data sharded on > the > > > > basis > > > > >> of an implicit route parameter. We send a search http request on > > > > google's > > > > >> network load balancer which divides requests amongst the 8 solr > > nodes. > > > > >> > > > > >> *Problem with this strategy:* > > > > >> If solr on any one of the nodes is down, the requests that come to > > > this > > > > >> node give 5xx. > > > > >> > > > > >> We are thinking of other strategies like > > > > >> 1. adding 2 vanilla nodes to this cluster(which will contain no > > data) > > > > which > > > > >> will be used for aggregating and serving requests i.e. instead of > > > > sending > > > > >> requests from lb to the 8 nodes, we will be sending the requests > to > > > the > > > > new > > > > >> nodes which will send internal requests on other nodes and fetch > > > > required > > > > >> data. > > > > >> 2. Instead of dividing requests using a load balancer, we can use > > > > zookeeper > > > > >> to connect with solr cloud. > > > > >> > > > > >> Would these strategies work? Is there a more optimized way using > > which > > > > we > > > > >> can request on solr? > > > > >> > > > > > > > > > > > > > > > -- > > > > > Sincerely yours > > > > > Mikhail Khludnev > > > > > > > > > > > > > > -- > > > Sincerely yours > > > Mikhail Khludnev > > > > > > > > -- > Sincerely yours > Mikhail Khludnev >
