Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-23 Thread Jan Høydahl
In k8s it’s quite simple already. But if the SIP is done there will be similar operational benefits for manual installs as well. But I would not hold my breath, may never happen. Jan Høydahl > 23. nov. 2023 kl. 18:50 skrev Dmitri Maziuk : > > On 11/23/23 04:59, Jan Høydahl wrote: > >> **FUTU

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-23 Thread Dmitri Maziuk
On 11/23/23 04:59, Jan Høydahl wrote: **FUTURE, NOT EXISTING**: With SIP-14 Solr may get a new "zookeeper" node-role that lets you spin up a solr Pod just for hosting an embedded ZK. FWIW I've always had better luck spinning up 2 separate jvms than running 2 things in one: IME linux has alwa

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-23 Thread Jan Høydahl
**NOW** In a k8s setting, ZK will have separate PODs (with both the suggested helm charts), and a POD hooks into the linux kernel to guarantee some resources so it is not starvated by hot Solr PODs on the same k8s node. So no need for separate machine, separate POD is ok. **FUTURE, NOT EXISTIN

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-23 Thread Radu Gheorghe
Hi Vince, Here's a tutorial for installing an external ZK + Solr on k8s: https://sematext.com/blog/solr-operator-autoscaling-tutorial/ Hope it helps. Radu -- Elasticsearch/OpenSearch & Solr Consulting, Production Support & Training Sematext Cloud - Full Stack Observability https://sematext.com/

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-23 Thread ufuk yılmaz
There’s a fundamental downside of embedded zk or putting zk on the same node as Solr. When there are many queries running, a Solr replica gets overloaded and it starts to affect the node itself (sometimes you can’t even ssh into the node when Solr is too busy), it also affects the Zk on the same

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Jan Høydahl
I think there is a product that speaks ZK api but stores everything in etcd :) Also, you may want to check out https://cwiki.apache.org/confluence/display/SOLR/SIP-18%3A+A+Solr+Kubernetes+Module+for+native+integration , an initiative that will let you use ConfigMaps for configSet files etc. I.e.

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Vince McMahon
Hi, Jan. Thanks for your links. They are super good. Luv it. On Wed, Nov 22, 2023 at 7:35 AM Jan Høydahl wrote: > Hi, > > I'd have a look at https://solr.apache.org/operator/ for more info about > the official support for running Solr in k8s. > > There is also this 3rd party Helm chart which i

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Vince McMahon
Ramesh, Can you elaborate more on the installation of ext. Zookeeper and how to make it work with solr on k8? On Wed, Nov 22, 2023, 6:26 AM Ramesh Balasubaramaniam wrote: > Yes, Possible. How many shards and replicas you are planning to create for > the 200 millions documents all based upon the

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread matthew sporleder
It's annoying that zookeeper is a hard dependency when etcd is just sitting there and all of the stuff in zk could easily fit into ConfigMaps. Ditto using a managed service like dynamodb instead of zookeeper :) On Wed, Nov 22, 2023 at 6:24 AM Ramesh Balasubaramaniam wrote: > > Yes, Possible. Ho

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Jan Høydahl
Hi, I'd have a look at https://solr.apache.org/operator/ for more info about the official support for running Solr in k8s. There is also this 3rd party Helm chart which is well maintained and perhaps easier to get started with: https://artifacthub.io/packages/helm/bitnami/solr Good luck. Jan

Re: Solr Cloud + zookeeper+ kubernetes setup

2023-11-22 Thread Ramesh Balasubaramaniam
Yes, Possible. How many shards and replicas you are planning to create for the 200 millions documents all based upon the index size ( how many stored fields ) and all. External zookeeper is best. SOLR in kubernetes setup also works well. Thanks, Ramesh On Wed, Nov 22, 2023 at 3:30 AM Vince McMah